㊩㈻教育未來發展之重要趨勢探討：
課程與教㈻
摘 要
醫學教育在課程與教學上的改革經過近幾十年來的發展，已在世界各 地建立了嶄新的教育模式。放眼未來，醫學教育將會擷取其他專業教育的特 質，將之融入其中 , 繼續進行創新並影響其他領域教育的發展。本文利用文 獻回顧，對於未來趨勢作一綜覽性的探討，包括： 1.有效的學習與傳授醫學 知識 , 技術與態度。2.模擬器學習的應用, 效果與其限制。3.定義臨床能力, 及 其對未來生涯發展大影響。 4.專業教育由畢業前延伸至畢業後, 強調持續與 終身學習的精神。 5.使用有效多元的能力評估工具。6.強調醫學倫理的 教育。最後我們提出醫學教育目標從醫師中心轉而以求診者為中心的典範 轉移以及醫學教育未來需面對的挑戰作為結論。
關鍵詞：模擬學習、臨床能力、醫學倫理、醫學教育、繼續教育 陸希平
嘉義聖馬爾定㊩院副院長及外科教授 國立嘉義大㈻及國立交通大㈻教授
吳毓瑩
國立台北教育大㈻心理與諮商㈻系教授
陳美足
嘉義聖馬爾定㊩院㊩療副院長
陳美惠
嘉義聖馬爾定㊩院院長
Abstract
A brand new model in medical education regarding curriculum and instruction has been created worldwide through continuous reform during these decades. Several excellent literature reviews and studies have given us important messages about the trends in the future development of medical education. In this article we focused on the following key issues. 1) locate the elements of effective learning; 2) adopt simulators--its application, effects and limitations; 3) define the medical professional competence--its importance in the future career development; 4) extend continuing and life long education--from pre-graduate to post-graduate; 5) utilize effective competences assessment tools; and 6) teach and assess medical ethics-- present status and future perspectives. Finally, paradigm shift of medical education curriculum from doctor-center to patient-center, as well as future challenges were discussed as concluding remarks.
Keywords: continuing education, medical education, medical ethics, medical professional competence, simulator in learning
Shi-Ping Luh
Vice Superintendent and Professor of Surgery, St Martin De Porres Hospital Professor of Chia-Yi and Chiao-Tung University
Trends in the Development of Medical Education:
Curriculum and Instruction
Yuh-Yin Wu
Professor, Department of Psychology and Counseling, National Taipei University of Education
Mei-Tsu Chen
Medical Deputy Superintendent, St Martin De Porres Hospital
Mei-Hui Chen
Superintendent, St Martin De Porres Hospital
Introduction
Medical educational reform in curriculum and instruction has been and is rapidly undergoing during the recent decades due to related political, technical, informational, economic, epidemiological changes. Such changes have impacts on: 1. Curriculum design from separated subjects to integrated curriculum. 2. Teaching style from tutor directed to collaborative learning with students. 3. Goal of learning emphasizing balance between the high technological medical resources and healthcare budge containment. 4. The ethics and law concept. 5. Professionalism along with knowledge and skills. 6. Shifting the concept from "cure only" for acute diseases to "care also" for chronic diseases. Based on these influences, learning styles were emphasized more toward active, collaborative, utilizing the informational technology, simulator provoking, case-based, and life long.
Against the above mentioned background, several excellent reviews and future perspectives presented by well-known speakers have given us important messages about the trends in the future development of medical education. Accompanying the literature reviews, six main issues were discussed through the article:
1.Locate the key elements in effective learning;
2.Adopt simulators in instruction--its application, effectiveness and limitations;
3.Define the medical professional competence--its importance in students’ career development;
4.Extend learning process as continuing and life long education--from pre-graduate to post-graduate;
5.Utilize assessment tools for competences evaluation;
6.Emphasize medical ethics--present status and future perspectives.
The future trend of medical education obviously is turning from doctor center to patient center. Medical educational objectives from the students /doctors competences to the patient outcomes base are both emphsized. However, reform is accompanying with challenges that we have to face in meeting and diecting the trends.
Future Trends
Elements of effective learning
Learning is a consequence of integrating new knowledge into existent knowledge structures. Learning new knowledge and skills is essential to solve new problems. Its related techniques can be improved by effortful and active attitudes. According to Norman (2000) and Norman and Schmidt (1992), learning effectiveness is related to learner’s prior knowledge, cognition character (a visual or verbal learner), capacities of multi-channels
cognition, and language gap. The cognitive process of learning includes recognition visual or verbal stimuli by eyes or ears. The sensory signal will be transferred into either working or long term memories. For example, multimedia presentation can provide the learners diversified modalities of stimuli. However, too many stimuli simultaneously can adversely influence the learning effectiveness. Unnecessary information will interfere, not amplify, learning. The limited capacities and channels of cognition make anything that diverts attention as an obstacle of effective learning .
Effective learning which could facilitate knowledge transfer should contain the following elements: contiguity, coherence, multiplicity, contrast, active participation, mixture of categories, and integration distribution (Saha, Poddar, & Mankad, 2005;
Thompson, Thompson, Kim, Krupa, & Shinkman, 1998). Learning materials should be correspondent with the goal mutually with contiguity and coherence. Thus, it is important to avoid seductive augmentation in teaching materials. Seductive augmentation means paying too much attention to others that distract the main issues, for example, using unrelated figures or cartoons. Mayer, Bailey, Mayer, Hillis, and Dvoracek (1987) found that picture with non-essential text was less effective in learning than picture only.
Regarding multiplicity, using multiple examples is better in knowledge transfer than single example with principles description by text or narration (Benbassat, Baumal, Borkan, & Ber, 2003). About contrast, learning through comparison of two cases will be more effective than that through studying one case and reading principles (Thomas, O'Connor, Albert, Boutain, & Brandt, 2001). Active learning/participation, as described above, can facilitate knowledge retention and its application (Michael, 2006). Norman (1988) and Norman and Schmidt (1992) have indicated that active learning can facilitate the retention and application of the knowledge. For example, problem or patient based learning (PBL), can be used to attain this goal. As for mixture of categories, examples containing both biological and practical ones can be memorized or applied much better than block of biological ones only . For example, the illustration of physiology about surface tension principle (to explain the motion of pulmonary alveoli), the LaPlace law (tension (T)= pressure (P) * radius (R)) is difficult to be understood (Fig. 1A). But if the figure is change to an iron block or cloths hung on the mid-point of a string, the principle becomes easier to follow (Fig. 1B). (When a phenomenon is difficult to explain, you can use an example in daily life to make it easier to be followed). This "mixture"
principle can also be applied in the mixture of examples and principles. Giving related examples immediately following a principle description (principle1-example 1-principle 2-example 2….). can be more effective to learning than giving all examples after all the principles being described (principle 1-principle 2….-example 1-example 2….)(mixture in categories) (Yang & Lewandowsky, 2004). Integrated learning can get better outcomes than separated learning (Requardt, 2006). Learning takes on multiple occasions can achieve
better effectiveness. For example, learning a topic 1 hour biweekly for 60 weeks would be better in retention and understanding than studying 6 hours per day for five consecutive days (Deiber, Missonnier, Bertrand, Gold, Fazio-Costa, Ibanez, & Giannakopoulos, 2007).
Figure 1. The explanation of LaPlace law.
(T: tension, P: pressure, R: radius, W: weight. the Fig A is a model of surface tension of a bubble; Fig B you can imagine your cloths (W) hang on a string)
Simulators in learning: Aapplication, effectiveness and limitations
Simulation is defined as a person device or set of conditions which attempts to present evaluation problems authentically (Issenberg, Petrusa, McGaghie, Felner, Waugh, Nash,
& Hart, 1999). Simulation originated from a war game named Wei-Hai about 500B.C.
in ancient China. In recent years, the development of information technology has made simulation broadly applied in many fields, such as architecture, engineering, law, aviation, military, medicine and research. The reasons why we use simulation are usually based on high risk or cost of the activity (such as war, aviation and medicine), the complexity of the activity (such as architecture, engineering), activity rare but important, and requiring team group collaboration (such as law, medicine) ("Military, police, law enforcement training,"
2006) Simulation in architecture has developed computerized design problems, judging proper size and sites allocation, as well as scoring system for architecture, including features (acceptable, indeterminate or unacceptable), feature clusters, vignettes (BP-3 or BT-6 cased), and pass or fail criteria ("NCARB: National Council of Architectural Registration Boards,2009"). Simulation in law has been applied in multi-state performance test in the U.S.. It included the ability of a legal affairs, description of the
A. All illustration from Medicine Laplace law T= P* R (difficult to follow)
B. A better model to explain it (easier)
Weight and string problem T= W/ 2sin � T large � T small
�
W W
court structure, factual information about the case, a memorandum of instructions, and a library containing the legal authorities. Examinees of JMLS - Multi-State Performance Test have 19 minutes to complete tasks and are evaluated his/her competences.
Simulation in aviation originated in 1970s by NASA to evaluate the knowledge and skills of commercial airline pilots and crews. According to their report ("Aviation Simulation Technology," 2009), around seventy percent of air-craft accidents were attributed to human factors, especially the communication errors. Simulation in military field in U.S.
and some European countries has developed a conceptual framework, including the score of the individual, team crew or unit (Erickson, 1985)
Simulation in medicine becomes more important because of increasing litigation originated from flaws in the physical examination and communication skills. In addition, more emphasis on patient privacy and right has limited the opportunity of training medical students and assessment of their competences on patient care (Hunt, 1968).
Simulation can provide medical students or trainees with limitless opportunities for highly reliable practice and variable difficulties according to the competences of the students, under well controlled stress-free environment. Simulator in learning facilitates knowledge integration, self learning, and corrective feedback (Dutta & Krummel, 2006).
In post-graduate continuing medical education (CME), simulation can be used to manage actual clinical conditions and teamwork collaborations (Dutta, Gaba, & Krummel, 2006).
Simulation can define clear outcomes, assess with objective rating scales on the checklist, and provide multiple learning styles, such as self-directed learning, demonstrative teaching, objective structured clinical exam, simulated bedside teaching, and simulated operations or invasive procedures (Dawson, Meyer, Lee, & Pevec, 2007). According to Roberts, Bell, and Duffy (2006), characteristics of well-constructed simulation include simulating behaviors with critical performance requirement, replicating salient features of real life, representing wide range of situations, and providing adequate feedback during and after the operation. Applying simulation in medicine, in the future, will be closely related to the development of virtual reality and digital imaging technologies (Carter, Schijven, Aggarwal, Grantcharov, Francis, Hanna, & Jakimowicz, 2005). There will be many scanning devices, in the near or far future, shared by medical team-members. Three dimensional holistic medicine conducted by digital physicians will be the future trend (Le, Lightner, VanderLei, Segura, & Gettman, 2007).
However, Maharaj, Ni Chonghaile, Higgins, Harte, and Laffey (2006) reminded that critical issues should be considered ahead of using this high-stake tool, such as its reliability & validity evaluation, benefits for certifying agencies and candidates, and quality of feedback to examiners and examinees.. As Carter et al. (2005), Le et al.(2007), and Maharaj et al.(2006) , varieties of challenges are ahead, such as difficulties in setting standards and scoring systems, interpreting and utilizing related data, reproducibility
of the data, evaluating cost-effectiveness, measuring reaction to stress or response to ethical dilemmas, and modifying score assessing because of the quantity diversity of the simulation.
Before the implementation of simulation in medical training or assessment, several questions should be asked (Carter et al., 2005; Le et al., 2007; Maharaj et al., 2006).
First of all, who are you doing it for? (pre-graduates or post-graduates ?). Secondly, what are the essential skills to be learned? We should notice whether simulation contains all critical components in actual conditions. Furthermore, how can you match the simulation elements to the learning skills? At last, how much fidelity complexity is enough? Since simulation is a high stake tool, evaluation of its cost-effectiveness is necessary.
A commonly used set of simulators in the medical school or teaching hospital, such as SimMen, can be used to train or assess the following skills, such as basic cardiorespirtory physiology (including pulses, heart/lung auscultation, etc.), basic skills (intubation, central or peripheral intravenous catheter insertion). This system is also capable to simulate rare situations to train coordination of team-members (emergency room, critical care unit, or operating room) (MacDowall, 2006; Nackman, Bermann, & Hammond, 2003).
Reviewing previous literature, the cost-effectiveness of simulation is high in coordinating anesthetic skills or teamwork collaboration, but low in clinical skill training or evaluation (for example, basic physiology or skills) (Carter et al., 2005; Le et al., 2007; MacDowall, 2006; Maharaj et al., 2006; Nackman et al., 2003). Therefore, the present applications of simulation are still limited in high stake testing, such as Objective Structured Clinical Examination (OSCE) , or training, such as Continuing Medical Education (CME) in American Board of Internal Medicine (ABIM) or Royal College of Canada Internal Medicine Certification Exam in Canada (Ross, Keay, Timmel, Alexander, Dignon, O’Mara, & O’Brien, 1999; Stefanidis, Sierra, Korndorffer, Dunne, Markley, Touchard, & Scott, 2006; Triola, Feldman, Kalet, Zabar, Kachur, Gillespie, & Lipkin, 2006). Different critical elements exist in different knowledge or skill, for example, 3-D dynamic view in gross anatomy; demonstration of hemo- or aero-dynamics in cardiopulmonary physiology; multiple models of heart sound coordinated with arterial and jugular pulses in physical examination of the circulatory system; manipulating instruments in cystoscopy, and touch feedback systems in laparscopy (Rolland, Wright,
& Kancherla, 1997; Triola et al., 2006). Simulation in gross anatomy with 3-D dynamic rotations is better than key views only for active and competent students, but not for passive and not-competent students (Richardson & Moody, 2000). Cardiacthoracic auscultation with Harvey simulator of high fidelity simulation could be used as a pre- and post- test for real patient practice (Anastakis, Regehr, Reznick, Cusimano, Murnaghan, Brown, & Hutchison, 1999; Issenberg, Gordon, & Greber, 2003). Analysis of six procedures from 23 studies was found the effectiveness of learning, ranging between the
text reading and cadaver dissection (Biasutto, Caussa, & Criado del Rio, 2006). However, further studies revealed that the beneficial effects of simulator or cadaver dissection are only marginal. Repeated practice might play a more important role than the high fidelity (Dunkin, Adrales, Apelgren, & Mellinger, 2007). Clements & Mackenzie (2005) reminded that in spite of high fidelity sometimes standing for better quality of learning, the increasing cost should also be considered.
Medical professional competences: Importance in the future career development Competence can be defined as the ability to practice with an adequate and safe standard. Synonymous terms for competence include sufficient, suitable, capable, legally qualified and fit for purpose (Epstein & Hundert, 2002). More holistic definitions of competence applied to medical practice have been mentioned as the habitual and judicious use of knowledge, technical skills, attitudes and reflection in the context of everyday practice (Zuberi, Bordage, & Norman, 2007). In other words, a doctor with qualified competences should be knowledgeable, skillful, dutiful and altruistic (Mukohara, Ban, Sobue, Shimada, Otani, & Yamada, 2006).
Delzell, Ringdahl, & Kruse (2005) classified the clinical skills matrix into knowledge (biology, clinical epidemiology, symptomatology), skills (medical procedures, communication, writing, computer literacy), information processing (evidence-based medicine, diagnostic tests, physical examination), decision making (logics and psychology), attitudes (toward patient, staff or science), and professionalism (accepting feedback, self motivating / organizing, ethics, virtue). Accreditation Council on General Medical Education (ACGME) defines core competence in patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism and system-based practice (Delzell et al., 2005). Institute for International Medical Education (2002) ("Global minimum essential requirements in medical education," 2002) divided seven domains of global essential requirements for doctors into core competences (information management, critical thinking, communication skills) and outskirt competences (clinical skills, scientific foundations, population health, professional values and attitudes). Harden (2006a) established a three circle model for the key abilitiy. The inner core (performance of tasks, doing the right things) is composed of what the doctor should do, such as history taking, physical examination, attitudes, interpersonal skills and other basic skills. The middle layer (approach to tasks, doing things right) is composed of how the doctor approaches the tasks such as scientific understanding, reasoning, symptomatology, presentation, evidence based medicine and other problem solving skills. The outer layer (professionalism, right person doing it) is composed of personal attributes of professionalism, such as decreasing the information asymmetry by explanation to your patients with efficacy, effectiveness,
and efficiency. The competence pyramid proposed by Curry and Makoul (1998) from lower to higher level, is " knows, know how, show how, and does" (Level 1 to 4). Level 1 (know) contains knowledge about the medical science. Level 2 (know how) includes tools for investigation and management. Level 3 (show) includes clinical skill practice, health promotion, communication skill and informed management. Level 4 (does) is composed of role model, attitude, ethics and decision making.
Professionalism has gradually become a new focus of medical education, in which only technical aspects were emphasized (Makoul, Curry, & Novack, 1998). American Board of Internal Medicine (ABIM) first defined professionalism as three commitments and six elements. The three commitments are: to the highest standards of excellence in the practice of medicine, to sustain the interests and welfare of patients, and to be responsive to the health needs of society. The elements include altruism, accountability, excellence, honor, integrity, and respect for others ("Learning objectives for medical student education--guidelines for medical schools: report I of the Medical School Objectives Project," 1999). Association of American Medical Colleges (AAMC) formulates four key attributes for medical students. Two of them (altruism and duty) are related to the professionalism ("Accreditation Council on General Medical Education (ACGME),"
2003). Accreditation Council on General Medical Education (ACGME) enlists professionalism as one of six core competences in resident training (Papadakis, Hodgson, Teherani, & Kohatsu, 2004).
All of above competences play important roles in the development of medical students. However, professionalism has been found as the only critical factor for future career success in reviewing previous literature. This is reasonable due to the fact that a doctor with professionalism can not only communicate with patients and judge their conditions well, but also possesses high motivation to pursue higher level of knowledge and skills. Most important of all, he (or she) encounters patients or other staffs with altruistic attitude, and thus he (or she) could not only be respected by others, but also be trusted to do the right things even under the situation that nobody monitors his (her) behavior. On the contrary, unprofessional behavior will result in need of more disciplinary actions in the future. A well validated case-control study from the Medical Board of California collected University of California, San Francisco(UCSF) graduates disciplined in board. They studied the correlation of various knowledge or skill indicators, including MCAT scores, undergraduate grade point average (GPA), medical school grades, USMLE step 1 score, as well as the evaluation of professionalism, with the disciplined actions in the future (Papadakis, Teherani, Banach, Knettler, Rattner, Stern, Veloski & Hodgson, 2005). The age, gender, board certification, MD education (USA/Canada vs not), and special fields were controlled in this study. Their results revealed that unprofessional behavior is the only significant risk factor to predict more (OR: 2.1 (1.2-4.4), p <0.01)
disciplinary actions or malpractice in the future. The other well validated case-control study also obtained similar results (Teherani, Hodgson, Banach, & Papadakis, 2005).
Unprofessional behaviors (OR 3.0 (1.9-4.8)) significantly increased the risk of future disciplinary action or malpractice when sex, GPA, MCAT score, and pass/fail in the Year 1-2 are partially controlled by matching group. Unprofessional behaviors include irresponsibility, poor self improvement, anxiety, insecurity, nervousness, immaturity, poor initiatives, and impaired relationship with peers. These are associated with future disciplinary actions or malpractice (Harden, 2006b; Khaliq, Dimassi, Huang, Narine,
& Smego, 2005). More importantly, this risk is cumulative, and thus unprofessional physicians without undergoing related continuing education will be increasingly risky in facing malpractice affairs or disciplinary actions (D. A. Davis, Thomson, Oxman, &
Haynes, 1995; Harden, 2006b).
Continuing and life-long learning: From pre-graduate to post-graduate
Outcome based education is not only beneficial to the pre-graduate student learning, but also needed for the post-graduate doctor re-education and training. Post-graduate doctors need structured specialty training to attain or retain general or special competences.
Post-graduate continuing medical education (CME) should also be provided to doctors to renew or integrate knowledge, as well as to improve their clinical skills (Davis, O'Brien, Freemantle, Wolf, Mazmanian, & Taylor-Vaisey, 1999). These post-graduate specialty trainings or post specialty continued education have to be effective because loading of competences training is heavy while time is very limited.
In reviewing related 136 literatures, specialty training or post-specialty continuing medical education is regarded to facilitate the imparity of knowledge to physicians, change physician’s attitude (22/26), acquire skills (12/15), change physician behavior (61/105), and changing clinical practice outcomes (14/30). Most of these references are not high in quality (Buttery, 2006; "Royal College of Physicians & Surgeons of Canada.,"
2003). Traditional CME outcomes measured by self report and peers evaluation reveal no significant effectiveness of the program (Buttery, 2006). More and more studies indicated that CME is not found to be useful in changing physician’s practice ("Royal College of Physicians & Surgeons of Canada.," 2003).
The traditional CME need some changes. It was found that traditional CME only focused on updating knowledge and doing little for skills, competence and quality of care. Continuing professional development (CPD), on the other hand, is the development of competences relevant to the quality of patient care. In addition to the traditional CME group learning activities, CPD contains new educational options, which can facilitate active learning and professionalism, such as: journal reading club, accredited self-assessment program, practice review and appraisal, structured learning projects,
and educational development (Tremblay, Drouin, Parker, Monette, Cote, & Reid, 2004).
There are 12 domains included in the CPD: local audits, hospital postgraduate meetings, college-based educational events, specialist society meetings, international meetings, regional /national audits, grand rounds & interdisciplinary meetings, professional development course, teaching and presentation meetings, examining postgraduates, and self-directed CPDs (Parboosingh & Gondocz, 1993). According to Boulay(2000) and Parboosingh & Gondocz (1993), evidences proved the effectiveness of CPD reflected in competences learning and transfer, development of professionalism, better communication skills,,bedside performance, and development of better learning styles particularly in active and life-long learning attitudes.
Effective competences assessment tools
Competence in medicine is defined as the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values and reflection in daily practice, for the benefit of the individuals and communities (Epstein
& Hundert, 2002). Competence stands for a habit, a contexture of general or contents specific ability. It can be developed as a life-long process (Klass, 2000; Leach, 2002).
Assessment for competences can be classified as formative and summative ones.
Formative assessments provide feedback in the process of learning, and thus it can guide future learning, promote reflection, and provide reassurance. Summative assessments provide the overall judgment, as well as qualify for advancement to higher levels of responsibility (Friedman Ben-David, 2000). Recently, the entry level assessment (pre-test) has been proved to facilitate the transition of competences for new-coming students or residents (Bansal, 2007). Goals of assessment as mentioned by Leung (2002) are as follows. At first, assessment can provide the learners a direction and motivation for future learning, including knowledge, skills and professionalism. Secondly, assessment of physicians can protect the public against possible harms from medical errors by upholding high professional standards and screening out trainees and physicians who are incompetent.
Contents of competence assessment can utilize the following related theories.
Mukohara et al.(2006) reported the Ban’s clinical skills matrix. Institute for International Medical Education (IIME) (2002) reported the seven domains of global essential requirements for doctors. Harden (2006) reported the three circles model. The Rethans et al (2002) reported the competence pyramid. The most popularly used of all is the Accreditation Council for Graduate Medical Education (ACGME), which defined core competence (Delzell et al., 2005) as patient care, medical knowledge, practice-based learning and improvement (PBLI), interpersonal and communication skills, professionalism and system-based practice.
Tools of competences assessment include written examination, supervising clinician
’s evaluation, direct observation or video review, clinical simulation, multisource (360-degree) assessments (by peers, patients and self), and portfolios (work place assessments) (Epstein, 2007). The choice of assessment tools depends on the characters of the competence because all of the above methods have their strengths and flaws. Five criteria can be used to determine the usefulness of a particular method of assessment (van der Vleuten, 1996): reliability (accurate and reproducible), validity (the assessment measure what it claims), impact on future learning and practice, acceptability to learners and faculty, and cost (trainee, institution)..
In general, written examination is more appropriate for the assessment of competences at the lower levels of the competence pyramid, i.e. level I & II: know, and know how.
Multiple choice questions (MCQs) is the most commonly applied written examination. It can assess the content, as well as the contexture knowledge if well designed. Examinees
’ performance can be reliably evaluated and graded by computer. However, designing high texture MCQs needs good training. Imperfect designing often results in cueing or unreal situations. MCQs are usually used as summative assessments including national in-service, licensing and certification examinations. The reliability and validity will attain satisfactory level if 175 to 250 items were included in a test (Case, 2000). Other written examinations, such as short-answer or structured essays, can test not only the content knowledge but also the knowledge of integration and appraisal, as well as the problem solving skills. They can also eliminate the cueing problems in MCQs. However, they are limited by the reliability and higher cost (time and money) for evaluation. Furthermore, imperfect designing still result in difficulty of transferring real life situations (Charlin, Roy, Brailovsky, Goulet, & van der Vleuten, 2000).
On the contrary, clinical simulation or portfolios are more suitable for the assessment of competences at higher levels of the competence pyramid (levels III & IV, i.e. show and does). Optimizing the use of assessment tools includes the following issues. Tutors can mix methods by a variety of environment to capture different aspects of performance.
They can also integrate the assessment into ongoing, contextual and developmental programs. Furthermore, they can use complex and structured assessment tools with balance; adopting validated methods (eg. direct observed behavior) or assessors (eg.
specialists). In addition, they can provide appropriate formative (comments for further improvement) and summative (pass or fail) feedback (Colliver, Vu, & Barrows, 1992;
Hakastian, 1971; Norman, Van der Vleuten, & De Graaff, 1991)
Direct observation by supervising clinicians, with either global or structured (eg.
mini-CEX) patterns, can be conducted in the ward round or teaching clinic. Learners will be assigned to take a patient history, perform physical examination, collect and interpret related laboratory data and image findings, and then make the diagnosis, differential
diagnosis and treatment plan. These tests can evaluate high texture knowledge, as well as clinical and communication skills. However, their applications are limited by time consuming, high cost, and lower reliability (existed some inter-rater inconsistency) (Pulito, Donnelly, Plymale, & Mentzer, 2006).
Objective structured clinical examinations (OSCE), which include standardized patients or high technology simulations, can formatively and summatively evaluate the integrated knowledge and skills, as well as the professionalism of the clerks, interns, or residents. This type of assessment is higher in validity than the written tests, and higher in reliability than the work place evaluation (due to standardized circumstances). Thus, it can either be applied as the national board examination or assessment of holistic competences.
However, this test is limited by high cost and time consuming (Tamblyn, 1998).
Multisource assessments (360-degree assessments) contains not only the work-place portfolios, but also the ratings from the peers, patients and self (Ramsey, Wenrich, Carline, Inui, Larson, & LoGerfo, 1993). Direct observation of procedure/skills (DOPS), Mini-CEX and patient satisfaction questionnaire (PSQ) are usually included in the assessment tools. Thus multisource assessment method can provide insight into trainees’
work habits, capacity for teamwork, and interpersonal sensitivity (Violato, Marini, Toews, Lockyer, & Fidler, 1997). Most importantly, as Norcini (2003) indicated that this mode of assessment can provide multisource feedback, either formative or summative, as well as narrative descriptions or numerical data. However, the limitations of this assessment are confidentiality of peer evaluation, bias from self evaluation and selection bias from the patients’ evaluation. Moreover, this is also a time and cost consuming activity (Eva &
Regehr, 2005).
Other new assessment tools included procedure-based assessments (PBAs ) for surgical operative skills. In United Kingdom, a number of basic and orthopedic procedures should be performed after related PBAs. Six domains of competences, such as consent, pre-operative planning and preparation, exposure and closure, intraoperative techniques, and post-operative management, are included in the PBAs evaluation.
The ratings of every items are scored as N (not appropriate), U (unsatisfactory) and S (satisfactory). Then the global summary can be classified as level 0 to 4 (insufficient evidence, unable, able under supervision, don’t need supervision, competent without supervision). This evaluation still need more evidences to prove its effectiveness. Patient satisfaction questionnaire in medical care (PSQ-care) in UK is designed with 10 questions and 30 patients are usually required for assessment to achieve satisfactory reliability and validity (Lawrence, Fauerbach, Heinberg, Doctor, & Thombs, 2006).
Mini-CEX evaluation for attending physicians usually contains 15 to 20 patient encounters and is administered during the routine practice in different settings of wards or clinics. Learners can be asked to carry out focused test for every encounter such as
family history or cardiovascular examination. They used 9 scored rating scales with two observers at each encounters. The competences being assessed include medical interview, humanistic, counseling skills, physical examination, clinical judgment, organization skill and efficiency. Corrigan, Bowden, & Owen (2006) and Kogan, Bellini, & Shea (2002) have found, it can effectively evaluate the clinical skills and provide instant feedback.
Teaching and assessing medical ethics: Present status and future perspectives
The progress in biomedical sciences and technology during the last few decades has been accompanied by the emergence of a broad range of ethical dilemmas. The direct doctor-patient relationship has also been transformed to a more complex process integrating the health-team, the healthcare seeker and the community. Coming of age of medical ethics made its education a basic requirement for all health professionals from undergraduate to postgraduate education (Daher, 2006; Iramaneerat, 2006). The goal of teaching medical ethics is to produce real ethical healthcare professionals, who possess a commitment to patient welfare, deal with complex uncertainty, and are reflexive, empathic and trustworthy (Campbell, 1989).
Teaching medical ethics encounters some questions as mentioned by London &
McCarthy (1998): the diversity of the teaching materials, the frequency and styles of teach, and the standard of failure in assessment. The attempt to integrate the values of medical ethics horizontally or vertically has been made in British Medical Association (BMA) Ethics Committee and Genetic Medical Council in UK. But they have also faced the difficulties of cultural differences (Gross, 2001). At present this consensus should be met by continuing communication based on mutual good and well recognizing others’
point of view (del Pozo & Fins, 2005). Teaching styles and assessments of medical ethics depend on the level of learning. The "learning pyramid" in ethics can be divided into three levels: from lower to higher are knowledge, habits and actions. In the knowledge level, some lectures about the ethics principles, guidelines or historical precedents are still required as the "core curriculum". Student-centered, problem-based courses about the ethics dilemmas could also be added to some extents. The recommended assessment tools at this stage are multiple choice questions (MCQs) or essays. In the habits level, the goals of learning are to cultivate their ethical awareness, critical thinking, and empathic attitude.
More student-centered and problem- based courses can be added. The recommended assessment tools at the stage are case reports or portfolios to test their responses to actual condition. In the actions level, the goal of learning is to possess the clinical ethical competency. The appropriate assessment tools are OSCE or Multisource (360-degree) assessments (Cobanoglu & Kayhan, 2006; Iramaneerat, 2006). Hundert, Hafferty, and Christakis (1996) have found that tutors’ training to formulate a "role model" (hidden curriculum) and pressures from peers also have large influences (either positive or
negative) in the effectiveness of learning.
A major impetus for teaching medical ethics is lack of institutional supports due to paucity of evidence about its effectiveness. Is this curriculum too small, less frequent or the students’ contact this course too late? Some recent studies have partially answered these questions and showed some effects of teaching medical ethics (Campbell, 1989;
Goldie, Schwartz, McConnachie, & Morrison, 2001; Herbert, 1997). Herbert (1997) found that teaching medical ethics is more effective in first to second years of students than the third to fourth years. The decreased ethical sensitivity of the later might due to lack of information. On the other hand, Jones, McCullough, and Richman (2005) found that ethics learning for the fourth or fifth year of students can facilitate their competences of knowledge, confidence, using ethics principles, breaking bad news and improving communication skills. Campbell (1989) found that ethical education can facilitate the students’ competences of moral reasoning (critical thinking, awareness of ethical issues, self knowledge of learning and applying principles. Goldie, Schwartz, McConnachie, and Morrison (2001) found the medical ethics learned by small group discussion can facilitate the first year medical students’ consciences and professional judgment (consent, autonomy and confidentiality). Yedidia, Gillespie, Kachur, Schwartz, Ockene, Chepaitis, Snyder, Lazare, and Lipkin (2003) proved that students undergoing ethics curriculum can be more competent in obtaining informed consents or communication with standardized patients. Malek, Geller, and Sugarman (2000) found more articulation and reasoning, as well as more suitable understanding in students who undergoing ethics training program.
At present, both the content and process of teaching medical ethics are now well documented. As ethics education comes of age, the development of consensus on core curriculum of ethics, as well as enrichment of instructional strategies with valid and clearly defined assessment methods based on outcome getting more and more important in the future.
Concluding Remarks
Paradigm shift of curriculum from doctore-center to patient-center
The development of evidence-based science has influenced many fields of medicine, including basic medicine, clinical medicine, as well as medical education.
Trustworthy research findings have been utilized as a basis for educational management and decision-making. Related evidence-based resources in medical education, such as web-based best evidence medical education (BEME), have replaced the personal opinions as the practice and implement guidelines in most medical schools. The recently developed concept of outcome-based researches has shifted the focus from outcomes of students’
performance (what doctors produced) to the outcomes of patient care quality (the ultimate goal) (Stearns, Stearns, Paulman, Chessman, Davis, Sherwood, Sheets, Steele, & Matson, 2007). Outcome-based researches on medical education mean that they should be focused on the outcomes of medical education, and thus the first should be answered if these studies have direct application to improve the ultimate outcomes, not only the students’
education, but more importantly, the patient care. The above goal is critical to either basic, applied, or evaluation researches
As described in the above paragraph, the levels of "outcome" can be divided into the individual, community, and population. Therefore, we should develop the assessment tools to evaluate not only "what doctors we produced", but also "does it make any differences in patient care". This paradigm shift as Thompson (2006) has pointed out that it seemed necessary because "patient centered" has become the most important goal for every field of medicine (clinical, basic, administrative, and education) at present time and in the future.
What should outcome-based mean to educator-researchers? There has been a new agenda for medical education with a reorientation from process to product (Collins, 2006).
As tests drive learning, we reasonably postulate that outcomes can drive curriculum or assessment. Researchers’ roles and related frameworks can be driven by outcomes to prioritize research questions to be better in patient care (Eva, Solomon, Neville, Ladouceur, Kaufman, Walsh & Norman, 2007).
However, outcome-based concept should not be overused because trying to capture everything by single concept usually means nothing. To avoid narrowing a perspective, sustained theoretical or innovative research is still very important. In addition, in spite of the curriculum design being outcome-based, appropriate flexibility is still required not to discourage the innovative minds of teachers or curriculum developers (Rees, Kavanagh, Harden, Shepherd, Brunton, Oliver, & Oakley, 2006). The implementation of outcome assessment also encountered several challenges, such as the uninvolved attitude or lack of staff development in some grass root level teachers, and ineffective communication or information with curriculum development in the administrative level.
Modification Based on Cultural Differences in Medical Education
The implementation of medical education in Taiwan should be modified according to its cultural characteristics. The grown-up backgrounds of our students are very different from those in the Euro-American countries where we learned the principles and practices of medical educational reform. Conflicts in behaviors, identities and values will occur among administrative staffs, tutors and students if we proceeded in these activities without modification. Furthermore, these students receiving non-modified models of medical education would encounter many conflicts in doctor-patient relationship or professional
performance, which would result in a huge obstacle for the development of their future careers.
Culture characteristics and related modifications in medical educational reforms in Taiwan can be summarized as follows.
First of all, Taiwan students tend to be silent in group meetings or discussions, because their culture told them "Silence is gold". Expressing individual opinion will be regarded as a negative impression by group members. Therefore courses requiring small group discussion, such as PBL, will be difficult to encourage them talking about their ideas. Under this situation, tutors should communicate the goals with the students, declare the principles and encourage the students to "speak out", no matter how ridiculous they say. Conducting and controlling the direction of discussion should be considered only after your students overcoming their fear and being able to talk liberally.
Secondly, Taiwan students are not so active in learning as Euro-American students.
They used to be achieving knowledge passively from their tutors in lecture. Therefore tutors should spend more time to give them the concepts and methods of "active learning".
Students with new concepts can utilize possible resources and then organize what they searched.
In the third, Taiwan students spent more time to memorize what they learned, and less time to think the reason why and how they can be applied clinically. Therefore tutors should fortify the students’ competences of reasoning, applying, or even challenging their learned knowledge. In addition, Taiwan students are not accustomed to judge things systematically. Tutors should conduct their students to cultivate and practice these competencies.
Furthermore, Students in Taiwan usually require more opportunities to practice their clinical skills. Tutors should encourage their students to expose themselves in actual or simulated clinical environments.
In addition, Tutors of medical education in Taiwan, with similar grown-up background to the students, should also receive education and training to change themselves. Since most of these tutors are also clinicians, they also should be dedicated in teaching their patients and families to create correct doctor-patient relationships.
Last of all, Shortage of resource and money is also an important issue in Taiwan medical educational reform. Thus we should keep in mind the concept of
"cost-effectiveness" and list the priority and the related cost investment during practical work.
Challenges for Medical Education
Curriculum design should meet the goals of medical education, such as knowledge integration, consistency and progression of competence, adult learning styles (reflection,
self directive and life-long learning) and professional thinking process (critical thinking or forward reasoning) through evidence--evidence based medicine (EBM) or best evidence on medical education (BEME). The trends of curriculum and instruction in medication has demonstrated the direction toward more ethical oriented, patient centered, simulator application, altruistic promoting, and redesigning instructional and assessment tools. Related studies showed that medical education is facing the challenges, such as knowledge explosion resulting difficulty in screening the valuable knowledge, differences in learning goals caused by diversified cultures, and inter-tutorial differences in the educational skills, personality characters (such as flexibility or sensitivity), or workloads.
Under the context of paradigm shifting from docter-center to patient-center, principles of effective curriculum planning as suggested by Fenderson (2005) include the followings. At first, curriculum should be redesigned with wide participation, not only the tutors, but also the students, patients and community members, with clarity of goals.
Secondly, academic leaders better share responsibility to assess the overall progression in outcomes of general and specific elements after its implementation. The maintaining and refining process of the curriculum should be continued by ongoing communication through workshops, seminars, and discussion among the institutes and community.
Selection of medical students suitable for future career is also a big challenge in the future of medical education. Ranking and determining the minimal standard of their exam performance, as well as the humanity characters (commitment, maturity, and communication skills) are the keys. On interviewing the candidates of medical students, we should know their age, background, commitments to medical practice, external pressure, experience and educational level, preparedness and maturity, communication and learning skills, as well as academic and social skills (Mallott, 2006).
Faculty development is another big issue to catch the trends as proposed. For example, the 360 degree assessment from students, staffs, patients and external evaluators can be used routinely and through the assists of capable faculty of informational technology (IT) backgraound (Eva & Regehr, 2005). Recommended strategies involve stimulating their responsibility by clear obligation, steps, accountabilities, well-designed implementation plan and establishing reward/punishment rules. These faculties should be supported, as well as assessed by the expert and peer groups. Related researches should be supported by grants to establish an excellent assessment system. It is believed that staff development can enhance skills development and mutual understanding..
The criteria as suggested by Mallott (2006) for effective staffs in medical education are as follows. They have to commit to active learning and clear communication.
Furthermore, They also have to work collaboratively to achieve the goals. They should encourage their students dedicated in learning from different teaching models. They also should apply validated principles and evaluate the quality of assessment. In addition, they
can recognize different agendas and cultures; and communicate well with teachers not only from the university hospital but also from the community and public health. Last but not the least of all, they are encouraged to have the ability to manage time to complete their educational activity and assessment in limited time.
Measuring learning outcomes effectively requires good assessment tools. Thus related staffs need more training of skills and should be based on agreed students and fairness.
Elements of a good assessment tool had better be valid, relevant, diverse, and more importantly, with the function of progressive achievement of goals or objectives. Effective evaluation is usually composed of summative evaluation which can assess the competence consistency of a student, as well the formative evaluation, which can give feedback during the learning process and provide a better way for further learning (Mallott, 2006).
The future trends in medical education emphasizes the principles of learner centered curriculum, education administration management by objectives, and care outcome based insturction. Therefore, setting goals based on students’ needs with ultimate patients’ care, creating proper curriculum and learning environments, as well as using multi-directional assessment tools will be the key elements and critical to improve quality of medical education.
致謝：本篇文章的完成感謝國科會研究計畫（94-2516-S-303-001）之支助以及嘉義聖馬爾
定醫院之行政支援。
Figure 2. Challenges for Medical Education
Challenges: consistency.
Student cohort Staff development Goals outcomes
Knowledge Skills Behaviors
Assess formation /summation Evaluation ongoing/terminal Curricular design: program or unit
Overall structure
Consists contents appropriate progression
Learning strategies Transimission / interaction (IT based) Self directed, practical perfo, problem solving skills
Texts readings.
References
Accreditation Council on General Medical Education (ACGME). (2003, 2003/5/28). from http://www.acgme.org/outcome/project/outintro.htm
ACPGBI: Training in coloproctology. (2006/12/20). from http://www.acpgbi.org.uk/traini ng/training.html.
Anastakis, D. J., Regehr, G., Reznick, R. K., Cusimano, M., Murnaghan, J., Brown, M., et al. (1999). Assessment of technical skills transfer from the bench training model to the human model. Am J Surg, 177(2), 167-170.
Aviation Simulation Technology, Inc. (2009/04/08). from http://www.ast-simulators.com/
site/index.html
Bansal, P. K. (2007). "Generalist" Medical Graduate to a "Specialty" Resident: Can an Entry Level Assessment Facilitate the Transition? Assessing the Preparedness Level of New Surgical Trainee. Paper presented at the the 4th APMEC in Singapore.
Benbassat, J., Baumal, R., Borkan, J. M., & Ber, R. (2003). Overcoming barriers to teaching the behavioral and social sciences to medical students. Acad Med, 78(4), 372-380.
Biasutto, S. N., Caussa, L. I., & Criado del Rio, L. E. (2006). Teaching anatomy: cadavers vs. computers? Ann Anat, 188(2), 187-190.
Buttery, C. M. (2006). Does continuing medical education work? BMJ, 333(7558), 99.
Campbell, A. V. (1989). On a principled medical ethic. Fam Pract, 6(2), 81-82.
Carter, F. J., Schijven, M. P., Aggarwal, R., Grantcharov, T., Francis, N. K., Hanna, G. B., et al. (2005). Consensus guidelines for validation of virtual reality surgical simulators.
Surg Endosc, 19(12), 1523-1532.
Case, S., & Swanson, D. (2000). Constructing written test questions for the basic and clinical sciences (3rd ed.). Philadelphia: National Board of Medical Examiners.
Charlin, B., Roy, L., Brailovsky, C., Goulet, F., & van der Vleuten, C. (2000). The Script Concordance test: a tool to assess the reflective clinician. Teach Learn Med, 12(4), 189-195.
Clements, R., & Mackenzie, R. (2005). Competence in prehospital care: evolving concepts. Emerg Med J, 22(7), 516-519.
Cobanoglu, N., & Kayhan, Z. (2006). An assessment of medical ethics education. Nurs Ethics, 13(5), 558-561.
Colliver, J. A., Vu, N. V., & Barrows, H. S. (1992). Screening test length for sequential testing with a standardized-patient examination: a receiver operating characteristic (ROC) analysis. Acad Med, 67(9), 592-595.
Corrigan, G., Bowden, F. J., & Owen, C. (2006). Mini-clinEx versus mini-CEX. Med Educ, 40(6), 599-600.
Curry, R. H., & Makoul, G. (1998). The evolution of courses in professional skills and perspectives for medical students. Acad Med, 73(1), 10-13.
Daher, M. (2006). Current trends in medical ethics education. J Med Liban, 54(3), 121-123.
Davis, D., O'Brien, M. A., Freemantle, N., Wolf, F. M., Mazmanian, P., & Taylor-Vaisey, A. (1999). Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA, 282(9), 867-874.
Davis, D. A., Thomson, M. A., Oxman, A. D., & Haynes, R. B. (1995). Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA, 274(9), 700-705.
Dawson, D. L., Meyer, J., Lee, E. S., & Pevec, W. C. (2007). Training with simulation improves residents' endovascular procedure skills. J Vasc Surg, 45(1), 149-154.
Deiber, M. P., Missonnier, P., Bertrand, O., Gold, G., Fazio-Costa, L., Ibanez, V., et al.
(2007). Distinction between perceptual and attentional processing in working memory tasks: a study of phase-locked and induced oscillatory brain dynamics. J Cogn Neurosci, 19(1), 158-172.
del Pozo, P. R., & Fins, J. J. (2005). The globalization of education in medical ethics and humanities: evolving pedagogy at Weill Cornell Medical College in Qatar. Acad Med, 80(2), 135-140.
Delzell, J. E., Jr., Ringdahl, E. N., & Kruse, R. L. (2005). The ACGME core competencies: a national survey of family medicine program directors. Fam Med, 37(8), 576-580.
du Boulay, C. (2000). From CME to CPD: getting better at getting better? BMJ, 320(7232), 393-394.
Dunkin, B., Adrales, G. L., Apelgren, K., & Mellinger, J. D. (2007). Surgical simulation:
a current review. Surg Endosc, 21(3), 357-366.
Dutta, S., Gaba, D., & Krummel, T. M. (2006). To simulate or not to simulate: what is the question? Ann Surg, 243(3), 301-303.
Dutta, S., & Krummel, T. M. (2006). Simulation: a new frontier in surgical education. Adv Surg, 40, 249-263.
Epstein, R. M. (2007). Assessment in medical education. N Engl J Med, 356(4), 387-396.
Epstein, R. M., & Hundert, E. M. (2002). Defining and assessing professional competence. JAMA, 287(2), 226-235.
Erickson, S. A. J. (1985). Fusing AI and simulation in military modeling. Paper presented at the Artifical intelligence in simulation conference.
Eva, K. W., & Regehr, G. (2005). Self-assessment in the health professions: a reformulation and research agenda. Acad Med, 80(10 Suppl), S46-54.
Eva, K. W., Solomon, P., Neville, A. J., Ladouceur, M., Kaufman, K., Walsh, A., et al.
(2007). Using a sampling strategy to address psychometric challenges in tutorial-based assessments. Adv Health Sci Educ Theory Pract, 12(1), 19-33.
Fenderson, B. A. (2005). Strategies for teaching pathology to graduate students and allied health professionals. Hum Pathol, 36(2), 146-153.
Friedman Ben-David, M. (2000). The role of assessment in expanding professional horizons. Medical Teacher, 22(5), 472-477.
Global minimum essential requirements in medical education. (2002). Med Teach, 24(2), 130-135.
Goldie, J., Schwartz, L., McConnachie, A., & Morrison, J. (2001). Impact of a new course on students' potential behaviour on encountering ethical dilemmas. Med Educ, 35(3), 295-302.
Gross, M. L. (2001). Medical ethics education: to what ends? J Eval Clin Pract, 7(4), 387-397.
Hakastian, R. A. (1971). The effects of type of examination anticipated on test preparation and performance. The Journal of Educational Research, 64, 319-324.
Harden, R. M. (2006a). International medical education and future directions: a global perspective. Acad Med, 81(12 Suppl), S22-29.
Harden, R. M. (2006b). Trends and the future of postgraduate medical education. Emerg Med J, 23(10), 798-802.
Herbert, C. L. (1997). 'To be or not to be'--an ethical debate on the not-for-resuscitation (NFR) status of a stroke patient. J Clin Nurs, 6(2), 99-105.
Hundert, E. M., Hafferty, F., & Christakis, D. (1996). Characteristics of the informal curriculum and trainees' ethical choices. Acad Med, 71(6), 624-642.
Hunt, E. (1968). Computer simulation: artificial intelligence studies and their relevance to psychology. Annu Rev Psychol, 19, 135-168.
Iramaneerat, C. (2006). Moral education in medical schools. J Med Assoc Thai, 89(11), 1987-1993.
Issenberg, S. B., Gordon, M. S., & Greber, A. A. (2003). Bedside cardiology skills training for the osteopathic internist using simulation technology. J Am Osteopath Assoc, 103(12), 603-607.
Issenberg, S. B., Petrusa, E. R., McGaghie, W. C., Felner, J. M., Waugh, R. A., Nash, I. S., et al. (1999). Effectiveness of a computer-based system to teach bedside cardiology.
Acad Med, 74(10 Suppl), S93-95.
JMLS - Multi-State Performance Test. (2006/07/18). from http://www.jmls.edu/events/M ulti-State.asp
Jones, J. W., McCullough, L. B., & Richman, B. W. (2005). Truth-telling about terminal diseases. Surgery, 137(3), 380-382.
Khaliq, A. A., Dimassi, H., Huang, C. Y., Narine, L., & Smego, R. A., Jr. (2005).
Disciplinary action against physicians: who is likely to get disciplined? Am J Med, 118(7), 773-777.
Klass, D. (2000). Reevaluation of clinical competency. Am J Phys Med Rehabil, 79(5), 481-486.
Kogan, J. R., Bellini, L. M., & Shea, J. A. (2002). Implementation of the mini-CEX to evaluate medical students' clinical skills. Acad Med, 77(11), 1156-1157.
Lawrence, J. W., Fauerbach, J. A., Heinberg, L. J., Doctor, M., & Thombs, B. D. (2006).
The reliability and validity of the Perceived Stigmatization Questionnaire (PSQ) and the Social Comfort Questionnaire (SCQ) among an adult burn survivor sample.
Psychol Assess, 18(1), 106-111.
Le, C. Q., Lightner, D. J., VanderLei, L., Segura, J. W., & Gettman, M. T. (2007). The current role of medical simulation in american urological residency training programs:
an assessment by program directors. J Urol, 177(1), 288-291.
Leach, D. C. (2002). Competence is a habit. JAMA, 287(2), 243-244.
Learning objectives for medical student education--guidelines for medical schools: report I of the Medical School Objectives Project. (1999). Acad Med, 74(1), 13-18.
Leung, W. C. (2002). Competency based medical training: review. Bmj, 325(7366), 693-696.
London, L., & McCarthy, G. (1998). Teaching medical students on the ethical dimensions of human rights: meeting the challenge in South Africa. J Med Ethics, 24(4), 257-262.
MacDowall, J. (2006). The assessment and treatment of the acutely ill patient--the role of the patient simulator as a teaching tool in the undergraduate programme. Med Teach, 28(4), 326-329.
Maharaj, C. H., Ni Chonghaile, M., Higgins, B. D., Harte, B. H., & Laffey, J. G. (2006).
Tracheal intubation by inexperienced medical residents using the Airtraq and Macintosh laryngoscopes--a manikin study. Am J Emerg Med, 24(7), 769-774.
Makoul, G., Curry, R. H., & Novack, D. H. (1998). The future of medical school courses in professional skills and perspectives. Acad Med, 73(1), 48-51.
Malek, J. I., Geller, G., & Sugarman, J. (2000). Talking about cases in bioethics: the effect of an intensive course on health care professionals. J Med Ethics, 26(2), 131-136.
Mallott, D. (2006). Interview, Dean's Letter, and Affective Domain Issues. Clin Orthop Relat Res, 449, 56-61.
Mayer, P. S., Bailey, G. H., Mayer, R. J., Hillis, A., & Dvoracek, J. E. (1987). Locative inferences in medical texts. J Med Syst, 11(2-3), 123-135.
Michael, J. (2006). Where's the evidence that active learning works? Adv Physiol Educ, 30(4), 159-167.
Military, police, law enforcement training,TSV. (2006, 2006/12/03). from
http://www.bioval.net/
Mukohara, K., Ban, N., Sobue, G., Shimada, Y., Otani, T., & Yamada, S. (2006). Follow the patient: process and outcome evaluation of medical students' educational experiences accompanying outpatients. Med Educ, 40(2), 158-165.
Nackman, G. B., Bermann, M., & Hammond, J. (2003). Effective use of human simulators in surgical education. J Surg Res, 115(2), 214-218.
NCARB: National Council of Architectural Registration Boards. (2009/04/09). from http://www.ncarb.org/
Norcini, J. J. (2003). Peer assessment of competence. Med Educ, 37(6), 539-543.
Norman, G. R. (1988). Problem-solving skills, solving problems and problem-based learning. Med Educ, 22(4), 279-286.
Norman, G. R. (2000). The epistemology of clinical reasoning: perspectives from philosophy, psychology, and neuroscience. Acad Med, 75(10 Suppl), S127-135.
Norman, G. R., & Schmidt, H. G. (1992). The psychological basis of problem-based learning: a review of the evidence. Acad Med, 67(9), 557-565.
Norman, G. R., Van der Vleuten, C. P., & De Graaff, E. (1991). Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability. Med Educ, 25(2), 119-126.
Papadakis, M. A., Hodgson, C. S., Teherani, A., & Kohatsu, N. D. (2004). Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board. Acad Med, 79(3), 244-249.
Papadakis, M. A., Teherani, A., Banach, M. A., Knettler, T. R., Rattner, S. L., Stern, D.
T., et al. (2005). Disciplinary action by medical boards and prior behavior in medical school. N Engl J Med, 353(25), 2673-2682.
Parboosingh, I. J., & Gondocz, S. T. (1993). The Maintenance of Competence (MOCOMP) Program: motivating specialists to appraise the quality of their continuing medical education activities. Can J Surg, 36(1), 29-32.
Pulito, A. R., Donnelly, M. B., Plymale, M., & Mentzer, R. M., Jr. (2006). What do faculty observe of medical students' clinical performance? Teach Learn Med, 18(2), 99-104.
Ramsey, P. G., Wenrich, M. D., Carline, J. D., Inui, T. S., Larson, E. B., & LoGerfo, J.
P. (1993). Use of peer ratings to evaluate physician performance. JAMA, 269(13), 1655-1660.
Rees, R., Kavanagh, J., Harden, A., Shepherd, J., Brunton, G., Oliver, S., et al. (2006).
Young people and physical activity: a systematic review matching their views to effective interventions. Health Educ Res, 21(6), 806-825.
Requardt, H. (2006). Technology solutions for better outcomes: integrated information management in key to productivity increases in medicine. Br J Radiol, 79(937), 17-23.
Rethans, J. J., Norcini, J. J., Baron-Maldonado, M., Blackmore, D., Jolly, B. C., LaDuca, T., et al. (2002). The relationship between competence and performance: implications for assessing practice performance. Med Educ, 36(10), 901-909.
Richardson, T. R., & Moody, J. M., Jr. (2000). Bedside cardiac examination: constancy in a sea of change. Curr Probl Cardiol, 25(11), 783-825.
Roberts, K. E., Bell, R. L., & Duffy, A. J. (2006). Evolution of surgical skills training.
World J Gastroenterol, 12(20), 3219-3224.
Rolland, J. P., Wright, D. L., & Kancherla, A. R. (1997). Towards a novel augmented-reality tool to visualize dynamic 3-D anatomy. Stud Health Technol Inform, 39, 337-348.
Ross, D. D., Keay, T., Timmel, D., Alexander, C., Dignon, C., O'Mara, A., et al. (1999).
Required training in hospice and palliative care at the University of Maryland School of Medicine. J Cancer Educ, 14(3), 132-136.
Royal College of Physicians & Surgeons of Canada. (2003, 2007/02/12). from http://www .rcpsc.medical.org
Royal College of Physicians. (2003, 2006/11/23). from http://www.rcplondon.uk
Saha, A., Poddar, E., & Mankad, M. (2005). Effectiveness of different methods of health education: a comparative assessment in a scientific conference. BMC Public Health, 5, 88.
Stearns, J. A., Stearns, M. A., Paulman, P. M., Chessman, A. W., Davis, A. K., Sherwood, R. A., et al. (2007). Family Medicine Curriculum Resource Project: the future. Fam Med, 39(1), 53-56.
Stefanidis, D., Sierra, R., Korndorffer, J. R., Jr., Dunne, J. B., Markley, S., Touchard, C.
L., et al. (2006). Intensive continuing medical education course training on simulators results in proficiency for laparoscopic suturing. Am J Surg, 191(1), 23-27.
Steinert, Y., Mann, K., Centeno, A., Dolmans, D., Spencer, J., Gelula, M., et al. (2006).
A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Med Teach, 28(6), 497-526.
Tamblyn, R. M. (1998). Use of standardized patients in the assessment of medical practice. CMAJ, 158(2), 205-207.
Teherani, A., Hodgson, C. S., Banach, M., & Papadakis, M. A. (2005). Domains of unprofessional behavior during medical school associated with future disciplinary action by a state medical board. Acad Med, 80(10 Suppl), S17-20.
Thomas, M. D., O'Connor, F. W., Albert, M. L., Boutain, D., & Brandt, P. A. (2001).
Case-based teaching and learning experiences. Issues Ment Health Nurs, 22(5), 517-531.
Thompson, J. W. (2006). Patient-centered care. J Am Acad Orthop Surg, 14(12), 636;
author reply 637.
Thompson, R. F., Thompson, J. K., Kim, J. J., Krupa, D. J., & Shinkman, P. G. (1998).
The nature of reinforcement in cerebellar learning. Neurobiol Learn Mem, 70(1-2), 150-176.
Tremblay, G. J., Drouin, D., Parker, J., Monette, C., Cote, D. F., & Reid, R. D. (2004).
The Canadian Cardiovascular Society and knowledge translation: turning best evidence into best practice. Can J Cardiol, 20(12), 1195-1198.
Triola, M., Feldman, H., Kalet, A. L., Zabar, S., Kachur, E. K., Gillespie, C., et al. (2006).
A randomized trial of teaching clinical skills using virtual and live standardized patients. J Gen Intern Med, 21(5), 424-429.
van. der. Vleuten,C. (1996). The assessment of professional competence: developments, research and practical implications. Advances in Health Sciences Education : Theory and Practice, 1, 41-67.
Veloski, J., Boex, J. R., Grasberger, M. J., Evans, A., & Wolfson, D. B. (2006). Systematic review of the literature on assessment, feedback and physicians' clinical performance:
BEME Guide No. 7. Med Teach, 28(2), 117-128.
Violato, C., Marini, A., Toews, J., Lockyer, J., & Fidler, H. (1997). Feasibility and psychometric properties of using peers, consulting physicians, co-workers, and patients to assess physicians. Acad Med, 72(10 Suppl 1), S82-84.
Yang, L. X., & Lewandowsky, S. (2004). Knowledge partitioning in categorization:
constraints on exemplar models. J Exp Psychol Learn Mem Cogn, 30(5), 1045-1064.
Yedidia, M. J., Gillespie, C. C., Kachur, E., Schwartz, M. D., Ockene, J., Chepaitis, A.
E., et al. (2003). Effect of communications training on medical student performance.
JAMA, 290(9), 1157-1165.
Zuberi, R. W., Bordage, G., & Norman, G. R. (2007). Validation of the SETOC instrument -- Student evaluation of teaching in outpatient clinics. Adv Health Sci Educ Theory Pract, 12(1), 55-69.
