Virtual Reality Simulator for Osteotomy and Fusion Involving the Musculoskeletal System

(1)

Virtual reality simulator for osteotomy and fusion

involving the musculoskeletal system

Ming-Shium Hsieh

a,

_{*, Ming-Dar Tsai}

b

_{, Wen-Chien Chang}

c

a_{Department of Orthopedics and Traumatology, Taipei Medical University Hospital, Taipei Medical University,}

252 Wu Hsing Street, Taipei 11031, Taiwan, ROC

b_{Institute of Information and Computer Engineering, Chung Yuan Christian University, Chung Li 32023, Taiwan, ROC} c_{Oral and Maxillofacial Division, Department of Dental, Taipei Medical University Hospital, Taipei, Taiwan, ROC}

Received 26 June 2001; accepted 5 September 2001

Abstract

In this study, the three-dimensional virtual reality simulation system described herein provides preoperative simulation to verify that the osteotomy and fusion procedures chosen to treat musculoskeletal defects are appropriate. The system also provides an excellent means of training surgeons in new operations without putting patients at risk, and may be especially useful for dif®cult surgical procedures often performed in orthopedics, craniofacial disease, or plastic and reconstructive surgery departments. The system can be used to teach intern and train resident doctors, and is a planning tool for visiting staff. q 2002 Elsevier Science Ltd. All rights reserved.

Keywords: Osteotomy and fusion; Three-dimensional environment; Surgery simulation; Virtual reality; Teaching, training and planning

1. Introduction

Over the past few years, computer graphics techniques using medical volumes have been widely used for three-dimensional (3D) simulated reconstruction of internal structures of the human body [1±5]. Virtual reality (VR)-based surgical simulation systems can provide valuable information for preoperative evaluation and planning. They are also useful for training, allowing students, residents, and surgeons, not familiar with a new technique, to gain experience before operating on real patients. Simulation systems that employ VRtechniques provide users 3D visual and, in some cases, even auditory or tactile environments, as well as 3D input tools that allow surgeons interacting with virtual patients to obtain more realistic results [6±10]. VRsimulation systems provide more accu-rate results and have a wider variety of uses than various commercial two-dimensional (2D) X-ray projection-based simulation systems [11,12].

The medical volume data used in 3D simulation systems are obtained through imaging techniques such as computed tomography (CT) and magnetic resonance (MR) imaging. These techniques provide a series of parallel cross-sectional

images; the image data are then represented as a volume (3D regular spatial array). Each volume element, or voxel, repre-sents a rectangular cuboid and is associated with a scalar value.

Commercial volume-based 3D graphics systems provide 3D images and `clipping' functions (simple planar, cylind-rical, or spherical cuts) that allow surgeons to remove obscuration or simulate simple surgical procedures. How-ever, more complicated procedures such as osteotomies, used to solve functional and aesthetic problems caused by skeletal deformities, are dif®cult to simulate.

A number of osteotomy and fusion operations carry very high failure rates: 2.4±20% for osteotomy of the mandible and maxilla [13±16]; 10±20% for high tibia osteotomy [17±19]; and 5±15% for anterior fusion of the spine [20±25]. The main causes of failure are poor anatomic section lines, poor contact surfaces, inappropriate size and shape of bone graft, and improper reduction position. These problems are present even when 2D surgical simulation is used to plan the procedure.

There are few studies in VRof the musculoskeletal system and they try to simulate the diagnosis of the musculoskeletal system, preoperative planning, and cranio-facial surgical procedures [26,27]. Meanwhile, the study of VRtechnique in orthopedic research and practice is recon-structed and studied by the intact cadavers imaged using CT, MRI and cryo-sectioning techniques [28].

Computerized Medical Imaging and Graphics 26 (2002) 91±101

PERGAMON

Computerized Medical Imaging

and Graphics

www.elsevier.com/locate/compmedimag

* Corresponding author. Tel.: 27372181x3118; fax: 1886-2-27375618.

(2)

We have developed a 3D VRvolume simulation system that can visualize conventional one-axial and multi-axial cross-sectional images [29,30]. This system can be used to simulate procedures, such as osteotomy of the musculo-skeletal system, that are used to correct deformities of the bones and joints commonly seen in orthopedics, oral and maxillofacial, dental, and plastic surgery departments [31]. In our system, the graphics output can be in the form of 3D or stereographic images that the user observes with shut-tered glasses. The user can use surgical instruments attached with a tracker to operate on the stereographic images [32] (Fig. 1).

Using this VRsurgical simulation system, we studied the deformities and instabilities of the knees, facial bones, and spine. Our system can simulate the actual procedures used during osteotomy and fusion, including sectioning, recog-nizing separate anatomic structures, removing, reposition-ing, fusreposition-ing, and healing of the structures and associated soft tissues. During simulation, the system can provide stereo-graphic images of skin, soft tissue, and bone surfaces to precisely predict the outcome of every step and the whole procedure. This preoperative veri®cation allows the surgeon to know whether the selected procedure can correct the deformity satisfactorily or not.

In this preliminary study, we evaluated the usefulness of our 3D VRsystem for surgical planning and prediction of outcome in three patients scheduled to receive dif®cult musculoskeletal procedures. To establish the range of applicability in terms of teaching, we asked four surgeons with different levels of experience to perform each simulation.

The purpose of the research is: (a) to verify the effective-ness of the system, whether the system can predict well the procedures of osteotomy and fusion, (b) to test the accuracy or validity of simulation procedure of simulator by some operation procedure under different levels of surgeons. 2. Subjects and methods

2.1. The 3D VR simulation system

The personal computer of the 3D VRsystem uses a Pentium-II 400 MHz processor, 256 MB RAM, and a 3D graphics accelerator (Gloria XXL by ELAS Inc., Aachen, Germany) and has a 21 in. monitor, shutter eyeglasses (Crystal Eyes PC by Stereo Graphics Inc., San Rafael, CA, USA), and a tracker (Inside TRAK by Polhemus Inc., Colchester, VT, USA).

The simulation functions of the 3D VRsystem are summarized brie¯y later. The tracker is attached to one end of the surgical instrument being used; the movement of the other end of the instrument is computed according to the position and angular attitude (a) of the tracker, and the length of the instrument (S) (Fig. 2A). Triangles are used to approximate the swept surface of the instrument. As can be seen in Fig. 2A, we can obtain two approximate triangles by connecting opposite ends of two consecutive instrument positions.

For simulating the musculoskeletal surgery, we use boundary pointers to represent and simulate boundary changes of bone structures and soft tissue, then normalize

M.-S. Hsieh et al. / Computerized Medical Imaging and Graphics 26 (2002) 91±101 92

Fig. 1. Three-dimensional reality surgery simulation in a virtual environment. (A) The simulation system, equipped with shuttered glasses and a tracker. (B) A pair of stereographic images: one for each eye.

(3)

the values of the voxels so that the values do not depend on tissue type. Because the voxel values are normalized, repositioning of bone structures and soft tissues does not in¯uence the values of the surrounding voxels. Therefore, we can change the contents of voxels to simulate various surgical procedures. This system can (1) compute changes of soft tissues, together with the bones; (2) simulate section, recognition, translation, rotation, and removal of anatomic structures along arbitrary directions; and (3) simulate fusion and healing of bones and soft tissues.

Figs. 2±5 show how this 3D VRsystem simulates

osteotomy and fusion procedures including sectioning, translating, rotating, removing and fusing anatomic struc-tures, and healing soft tissues. The volumes used in these ®gures are 20 CT slices of a human skull. Fig. 2B shows the simulation of recognition and sectioning. It also shows the results of a series of simulation computations: generating two swept surfaces, computing intersections between voxels and the swept surfaces, and changing boundary pointers and values of the voxels at the intersections to represent the sections. Because the stereo visual perception by the shuttle glass may not be exact for somebody, the user may not section the correct position; therefore, the section simulation may fail if he sections the wrong position.

After sectioning, the system implements the recognition computation when the user wants to move or remove the structure. We use the seed and ¯ood algorithm to ®nd the voxels inside a set of sectioned boundaries. The seed and ¯ood algorithm is a recursive technique used to ®ll an area or volume where boundaries (endmost columns) have been drawn closed [33]. If the boundaries of the sectioned struc-ture do not form a closed area, the computation will ¯ood out of the side of the structure that still connects with the skeleton. Therefore, the recognition simulation may fail if the sectioned structure by the user cannot separate from the skeleton.

The sectioned structure can be translated by interchang-ing the contents of the voxels where the structure was and where it will be (Fig. 3A), or can be removed by deleting the contents of the structure voxels (Fig. 3B). When the struc-ture is repositioned, the system also implements a collision test by detecting whether the bone voxels are present between the new and old position of the structure. In Fig. 3A, the collision test shows collision of the sectioned structure with other bone voxels. The removing simulation may fail, if the structure is not yet recognized or a collision occurred. The translation simulation may also fail if the structure is not yet recognized or a collision occurs, or the new place to which the structure translates there exists another bone structure.

In the VRsimulation environment, the surgeon can also rotate the stereographic image to section and remove the anatomic structures in different frontal, lateral, or even

Fig. 3. (A) Translation simulation: collision test and moving computation after recognition of an anatomic structure (bone) (solid arrow: translating structure). (B) Removing the anatomic structure (hollow arrow: where structure was removed).

Fig. 2. Swept surface computed according to tracker position and section by swept surface. (A) Motion of surgical instrument simulated by the tracker.

a, the attitude of the tracker; S, the length of the surgical instrument. (B) Sectioning simulation: generating swept surfaces, computation of intersec-tions of swept surfaces and bones. Solid arrow: swept surface; hollow arrow: surgical instrument.

(4)

oblique views; the rotation computation is similar to the translation computation. The rotation simulation may also fail, if the structure is not yet recognized or a collision occurs, or another structure occupies the place the structure will be moved to.

Figs. 4 and 5 show the simulation of fusing separate structures and healing of associated soft tissues. For fusing bones, the user has to specify fusion surfaces on the two structures to be fused (Fig. 4A). The system generates callus bone voxels between the fusion surfaces, and recognizes the two structures as one continuous structure (Fig. 4B). The bone fusing simulation may fail, if the speci®ed surfaces on two fusing structures are not consistent.

To simulate healing of associated soft tissues, the user has to specify healing surfaces on the two soft tissues to be healed. The system then generates soft tissue voxels between the healing surface, and recognizes the two soft tissues as being continuous. Fig. 5 shows the results of simulated healing of soft tissues. The healing up simulation may fail, if the speci®ed surfaces on the soft tissue for healing are not consistent.

2.2. Actual and simulated procedures

Three patients scheduled to receive musculoskeletal operations were selected for this study. The indications for surgery were hypoplasia of the maxilla (retronathism) (patient 1), osteoarthritis of the knee with genus varus (patient 2), and instability of the lumbar vertebrae 4±5 (L4±5) with herniated intervertebral disk (HIVD) (patient

3). All patients were treated at Taipei Medical University Hospital during the period of January 1995±February 1997. The characteristics of the patients are summarized in Table 1. Informed consent was obtained from all patients and control subjects.

Patient 1, a 30-year-old man, had had a deformity of the mandible with malocclusion since birth, and was dissatis®ed with his facial appearance. Hypoplasia of the mandible was diagnosed, and further management with reconstruction was advised. The operation included bilateral corrective osteot-omy of mandible and bone fusion with bone graft.

Patient 2, a 73-year-old woman, presented with right knee pain associated with dif®culty in walking and deformity. The plain X-ray ®lm, clinical syndrome and signs, and CT images of the right knee led to the diagnosis of severe osteoarthritis with genus varus. The operation included corrective osteotomy of high tibia and bone fusion.

Patient 3, a 52-year-old woman, had a 2- to 3-year history of severe low back pain with mild right sciatica. Electro-physiologic study, functional views of plain lateral X-ray ®lms, and MRimages led to the diagnosis of instability (spondylolisthesis, grade I) with L-HIVD at L4±5 and L5±S1. Surgical intervention was indicated after conserva-tive treatment failed. The operation included decompression of the nerve root and cord, and anterior fusion of L4±5 with a bone graft after partial osteotomy of the L4±5 space.

Patients 1 and 2 were examined with a spherical CT system (Hispeed CT/i system, General Electric, Milwaukee, WI, USA). Patient 3 was examined in three-axial cross-sections (sagittal, transverse, and coronal) with a 0.5 T MRimaging machine (General Electric, Milwaukee, WI, USA).

The actual procedures were performed by the same senior surgeon. The clinical results after treatment were assessed according to four criteria: (1) at least moderate to complete pain relief, or no narcotic medication required; (2) minimal deformity or no need for cosmetics; (3) return to pre-injury functional status, or no abnormal appearance and normal functional status; and (4) patient satisfaction with the outcome of the procedure. The overall clinical outcome was thus rated as excellent (all four of the criteria met); good (three of the criteria met); fair (two of the criteria met); or poor (one or none of the criteria met). Patients

Fig. 5. Healing associated soft tissues (solid arrow: new formation of soft tissue; hollow arrow: bone structure).

Fig. 4. Fusing bone structures. (A) Before fusion (solid arrow: fusing structure; hollow arrow: fusing surfaces). (B) After fusing (solid star: callus formation; hollow star: structure after fusion).

(5)

with excellent or good results were considered to have successful treatment outcomes.

Four surgeons of orthopedic surgery with various levels of expertise conducted simulated surgical procedures on each of the three patients. The surgeons included one intern as a student, one resident as a beginner of training course, one chief resident as a junior surgeon, and one senior surgeon (visiting staff) as specialist. Each simulated proce-dure was performed independently by every surgeon. Two procedures were simulated for patient 1: mandible osteot-omy plus bone graft, and maxillary osteotosteot-omy plus fusion with repositioning. The two procedures may achieve the same purpose of correcting the deformity, therefore, each of them can be chosen to operate depending on operative habit of surgeons. Thus, there were a total of four procedures and 16 simulations.

The assessment of the results of simulated surgery included seven criteria (discrete steps): (1) sectioning; (2) recognition; (3) removing; (4) translating; (5) rotating; (6) fusing anatomic structures; and (7) healing up associated soft tissue. Characteristics of success step is ®nished over 80% by the simulation procedure in each step. The overall VRsimulation result was rated according to the number of steps performed successfully: excellent (®ve or more); good (four); fair (three); or poor (two or fewer). Simulations with excellent or good results were considered successful.

The success rate can be improved by repeating learning and training. However, actual procedure can only be per-formed by irreversible procedure under a well-trained senior surgeon.

3. Results

Clinically, all patients had satisfactory outcomes: patients 1 and 2 had excellent results and patient 3 had good results (Table 1). There were two women and one man, and the follow-up period of all patients was an average of 2.5 yr (range, 2.1±2.8 yr).

The visiting senior surgeon achieved excellent results in all four simulated procedures. The chief resident, resident, and intern all achieved excellent results in three of the simu-lated procedures, and good results in the other. The results of the individual steps of simulation are listed in Table 2. Thus, all simulated procedures had satisfactory and success-ful outcomes.

4. Case studies

The simulated and actual outcomes of the three patients are summarized in the following. Although our system provides a pair of stereographic images for every simulation procedure, we show only one image from each pair.

Table 1 Charact eristics, operative method s, and outcom es of patient s (VR, virtual real ity; M, male; CO, corrective osteoto my; FU, fusio n; BG, bone gr aft; E, exce llent; F, fema le; L, lumbar spine; G, good; H IVD, herniat ed intervertebral dis k; OA, ost eoarthriti s; N/A, not appl icable) Subject No. Age (yr ) Sex Clinical diagnosi s Site of operation VR-sim ulated opera tion Clin ical operati on Follow -up post-opera tion Clinical result s Patient 1 a 30 M Retronat hism Mand ible CO 1 FU 1 BG (procedur e I) C O 1 FU 1 BG 2Y8M E Maxill a C O 1 FU (p rocedure II) N/A N/A N/A 2 73 F OA with genus varus High tibia CO 1 FU CO 1 FU 2Y6M E 3 52 F Instab ility of L4 ±5 1 HIVD L4 ±5 (anterior) Anterior FU 1 BG Anteri or FU 1 BG 2Y1M G a Patient 1 received two simula ted proc edures: procedure I: bila teral co rrective osteoto my of man dible, and bone fusio n with bone graft; procedure II :corrective ost eotomy with restropositi on of maxillary and bone fusion, and actual ly received `pr ocedure I' operation.

(6)

4.1. Patient 1 Fig. 6(A )± (C )show sthe 3D images of the lateral view of the face, later al view of the skull, and frontal view of the skull before simul ation. The deform ity of the max illa, maxilla ry retrona thism, is evident . 4.1.1. Procedure I Fig. 7 shows the 3D images duri ng the simul ation of procedure I. The left mandible is ®rst sectioned. The M.-S. Hsieh et al. /Computerize d Medical Imaging and Graphics 26 (2002) 91 ±1 01 96 Table 2

Results of simulation of four musculoskeletal procedures, performed by four surgeons, with a 3D VR-based simulation system (P1, patient 1; P2, patient 2; P3, patient 3; PI, procedure I; PII, procedure II; (1) successful; (2) unsuccessful; E, excellent (®ve or more steps completed successfully); G, good (four steps completed successfully))

Criteria for assessment Procedure Surgeon Success rate

Intern Resident Chief resident Visiting surgeon

P1 P2 P3 P1 P2 P3 P1 P2 P3 P1 P2 P3

PI PII PI PII PI PII PI PII

1 Sectioning 2 1 1 2 2 1 1 1 2 1 1 1 2 1 1 1 11/36 2 Recognizing 1 1 2 1 1 1 1 2 1 1 1 2 1 1 1 1 13/16 3 Removing 1 2 1 2 1 2 1 1 1 2 1 2 1 2 1 1 10/16 4 Translating 2 1 1 1 2 1 1 1 1 1 2 1 1 1 1 1 13/16 5 Rotating 1 2 2 1 2 1 2 1 1 2 2 1 1 2 2 1 7/16 6 Fusing 1 1 1 1 1 2 1 2 1 2 1 1 1 1 1 1 13/16 7 Healing 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 15/16

Total number of successful steps 5 5 5 4 4 5 6 5 6 4 5 5 6 5 6 7 83/112

R esults E E E G G E E E E G E E E E E E Fig. 6. Three-dim ensional images before simulation (patient 1). (A) Lateral view of the face (solid arrow: hypoplasia of maxilla). (B) Lateral view of the skull (hollow arrow: hypoplasia of maxilla). (C) Frontal view of the skull (solid star: hypoplasia of maxilla).

(7)

sectioned bone is then recognized, and the bone segment is removed (Fig. 7A). The right mandible is also sectioned; the section is then recognized, and removed (Fig. 7B). Fig. 7C shows the frontal view of Fig. 7B. Two bone grafts are then inserted into the spaces, and fusing and healing simulation are implemented. Fig. 8A shows the results of fusion simu-lation, and Fig. 8B shows the results of healing simulation.

Comparing Fig. 8A and B with Fig. 6A and B, we can observe that the mandible has been moved forward and the deformity has been corrected.

4.1.2. Procedure II

Fig. 9 shows the simulation of the upper anterior sub-apical osteotomy, which moves the maxilla backward to correct the mandibular retrusion. In Fig. 9A, the surgeon has sectioned and removed the right upper premolar teeth and socket, and is in the process of sectioning the left premolar tooth and socket (red line). Fig. 9B shows the same process of sectioning the right premolar tooth and socket. In Fig. 9C, the left premolar tooth and socket have been sectioned, recognized, and removed.

After the two premolar structures are removed, the surgeon sections the nasal septum to separate the maxilla from the skull and reposition the maxilla backward 6 mm and upward 3 mm. Then, the surgeon lets the system heal the skull and maxilla. Fig. 9C shows the frontal view of the rendering results after the maxilla has been repositioned and fused with the skull.

Fig. 10A shows the rendering results after fusion, in the lateral view. Fig. 10B shows the rendering results after healing, in the lateral view. Comparing Fig. 10A and B

Fig. 7. Three-dimensional images during the simulation of bilateral correc-tive osteotomy of ramus of mandible with bone graft in patient 1. (A) The left mandible is sectioned, recognized, and removed (hollow arrow: section line of osteotomy of the ramus of the left side of the mandible). (B) The right mandible is also sectioned, recognized, and removed (solid arrow: section line of osteotomy of the ramus of the right side of the mandible). (C) Frontal view of post-bilateral osteotomy and anterior reposition of the mandible.

Fig. 8. (A) Simulated healing after the procedures shown in Fig. 7 (patient 1). (B) Simulated fusion after the procedures shown in Fig. 7. Comparing Fig. 8A and B with Fig. 6A and B, we can observe that the mandible has been moved forward and the deformity has been corrected.

(8)

with Fig. 6A and B, we can observe that the maxilla has been moved backward, and the deformity has been corrected. This preoperative veri®cation indicates that the procedure can correct the deformity satisfactorily.

4.2. Patient 2

Fig. 11A shows the CT image (256 £ 256 £ 26 voxels) of

the knee before simulated surgery. As the angular deviation of the anatomic axis of the femur and tibia is 148, the deformity is classi®ed as a type of osteoarthritis with genus varus. The surgeons simulated a high tibia osteotomy to correct this deformity, by sectioning horizontally to the tibia about 2.5 cm below the tibia plateau, then sectioning obliquely to form a wedge-shaped bone fragment (Fig. 11B). The tibia and ®bula were then rotated to correct the angular

Fig. 9. Three-dimensional images during the simulation of upper anterior subapical osteotomy, performed to move the maxilla backward to correct mandibular retrusion (patient 1). (A) Simulation of upper anterior subapical osteotomy. The right upper premolar tooth and socket have been removed, and the left premolar tooth and socket have been sectioned (solid arrow: osteotomy site on the right side of the maxilla; red line socket). (B) The left premolar tooth and socket are also sectioned, recognized, and removed (hollow arrow: osteotomy site on the left side of the maxilla). (C) Simulation of fusion of the sectioned and repositioned maxilla with the skull (solid star: repositioned and fused maxilla).

(9)

deformity, and the bone was fused (Fig. 11C). The small angular deviation after simulation (about 28) suggests that the proposed surgical procedures can achieve a satisfactory result.

4.3. Patient 3

Patient 3 required anterior fusion with vertebral osteot-omy and bone graft for instability of the lumbar spine. In this case, MRimage slices were used for VRsimulation. The whole simulation process is shown in the frontal view in Fig. 12A, and the lateral view in Fig. 12B. The results of the preoperative simulation indicated that anterior fusion of L4±5 with osteotomy of L4 and L5 plus autogenous bone graft with an iliac bone block graft would achieve the desired result.

5. Discussion

Traditional paper surgery and model surgery simulation techniques are not practical for simulations of osteotomy and fusion. Moreover, the entire preparatory process for implementing osteotomy and fusion operations is compli-cated, time-consuming, and imprecise. Our new system can solve these problems, because the system can provide inter-active simulated result for every step of surgical procedure. In our system, we compute the soft tissue changes as bones. This approach may not precise, because soft tissues are not rigid. However, simulating soft tissue changes is complicated. Usually, the ®nite element method must be applied for computing results precisely [32]. This approach

Fig. 10. Results of simulated upper anterior subapical osteotomy in patient 1. (A) Rendering results after healing, in the lateral view. (B) Rendering result after fusion, in the lateral view. Comparing Fig. 10A and B with Fig. 6A and B, we can see that the maxilla has been moved backward, and the deformity has been corrected.

Fig. 11. Three-dimensional images of simulated high tibia osteotomy of the right knee (patient 2). (A) Bone surface of the knee (frontal view). (B) Simulation of sectioning and removing the wedge-shaped bone fragment (arrow: wedge-shaped osteotomy). (C) Simulation of rotating the tibia and the ®bula to fuse with the proximal tibia.

(10)

becomes expensive in terms of computation. The reposi-tioned soft structures in corrective osteotomy are usually small. Therefore, we simplify the repositioning calculations by considering the associated soft tissues as rigid structures, and deal with these tissues as structures with separate volumes that are conjoined to the structure of interest.

The VRtechnique described herein can be helpful in the management of disease of the musculoskeletal system, such as for correction of deformity, instability, and functional impairment. The results of this study show it to be a feasible and practical tool for clinical and theoretical study.

Osteotomy and fusion involving the musculoskeletal system can be complicated and dif®cult procedures. The new VRsimulation technique provides preoperative plan-ning and practice, which may help in overcoming these dif®culties. Surgeons, students, as well as specialists, can perform the simulation until satisfactory results are achieved.

In the future, this system can be further developed as a

tool for diagnosis, management planning, and prognosis assessment. The addition of a hatpin function, by using force feedback devices, may improve its usefulness. 6. Summary

Simulation of osteotomy and musculoskeletal fusion is dif®cult with currently available surgical simulation techni-ques. In this study, we evaluate the usefulness of 3D simu-lation in a VRenvironment for simusimu-lation and preoperative planning of dif®cult osteotomy and musculoskeletal fusion procedures.

Three patients scheduled for osteotomy or musculo-skeletal fusions were recruited. One patient required osteot-omy and anterior fusion of the spine, one required high tibia osteotomy, and the other required corrective osteotomy of the maxilla and mandible. Four surgeons (one intern, one resident, one chief resident, and one visiting senior surgeon) used the new 3D VRsystem to simulate the surgical proce-dures, each of which required steps of sectioning, recogniz-ing, removrecogniz-ing, translatrecogniz-ing, rotatrecogniz-ing, and fusing of bone, as well as healing of anatomic structures and associated soft tissue. Two different operations were simulated for one of the patients; thus, 16 simulated and three actual procedures were performed. The results of simulation were compared with intraoperative ®ndings and postoperative outcomes.

Two of the patients had excellent outcomes and the other had a good outcome from the actual procedure. The success rates of each step of the simulations (de®ned as .80% completion of the procedure) were as follows: sectioning of bone, 11/16; recognition of computation, 13/16; remov-ing the sectioned structure, 10/16; translatremov-ing the structure, 13/16; rotating the structure, 7/16; fusion, 13/16; healing, 15/16. The four surgeons had similar success rates for each step.

In conclusions, this 3D VRsimulation technique appears to be a feasible and practical tool for simulation and planning of osteotomy and musculoskeletal fusion, and may be particularly useful for dif®cult procedures com-monly performed in orthopedics, craniofacial reconstruc-tion, and plastic surgery departments.

Acknowledgements

This study was partially sponsored by the National Science Council (NSC), Taiwan/ROC; grant numbers 86-2213-E033-036, 87-2213-E033-005, NSC-89-2320-B038- 019, NSC-89-2314-B038-057.

References

[1] Baillie J, Jowell P, Evangelou W, Bickel W. Use of computer graphics simulation for teaching of ¯exible sigmoidoscopy. Endoscopy 1991;23(3):126±9.

Fig. 12. Segmentation and surgery simulation in the frontal (A) and lateral (B) views (numbers refer to (A) and (B)) (patient 3). (1) Bone surfaces (white areas) of the lumbar spine, including disc spaces (green areas), and spinal cord and roots (red areas). (2) The same as in (1), after bone disarti-culation (hollow arrow: far lateral disc, protruded L5±S1 and L5 nerve root compression; solid arrow: central disc, extruded L4±5, and L4 nerve root compression). (3) Results of simulated anterior decompression, discect-omy, and partial corpectomy (osteotomy of the lower portion of L4, and upper portion of L5) (hollow star: instability of lumbar space, after anterior osteotomy (partial corpectomy). (4) Fusion with a suitable bone graft (from iliac bone) after osteotomy (solid star: bone graft).

(11)

[2] Caponetti L, Fanell AM. Computer-aided simulation for bone surgery. IEEE CG&A 1993;13(6):87±91.

[3] Cover SA, Ezquerra NF, O'Brien JF. Interactively deformable models for surgery simulation. IEEE CG&A 1993;13(6):68±75.

[4] Rosenquist B. Anterior segmental maxillary osteotomy. A 24-month follow-up. Int J Oral Maxillofac Surg 1993;22(4):210±3.

[5] Udupa JK, Odhner D. Fast visualization, manipulation and analysis of binary volumetric objects. IEEE CG&A 1991;11(6):53±63. [6] Bainville E, Chaffnjon P, Cinquin P. Computer generated visual

assistance during retroperitoneoscopy. Comput Biol Med 1995; 25(2):165±71.

[7] Hunter IW, Jones LA, Sagar MA, Lafontaine SR, Hunter PJ. Ophthalmic microsurgical robot and associated virtual environment. Comput Biol Med 1995;25(2):173±82.

[8] Kuhlen T, Dohle C. Virtual reality for physically disabled people. Comput Biol Med 1995;25(2):205±11.

[9] Ota D, Loftin B, Saito T, Lea R, Keller J. Virtual reality in surgical education. Comput Biol Med 1995;25(2):127±37.

[10] Ziegler R, Fischer G, Muller W, Gobel M. Virtual reality arthroscopy training simulator. Comput Biol Med 1995;25(2):193±203. [11] Bhatia SS, Sowray JH. A computer-aided design for orthognathic

surgery. Br J Oral Maxillofac Surg 1984;22(4):237±53.

[12] Cutting CB, Grayson B, Bookstein FL, Fellingham L, McCarthy JG. Computer-aided planning and evaluation of facial and orthognathic surgery. Clin Plast Surg 1986;13(3):449±62.

[13] Boleo TJ. Some ideas on relapse after remodeling of prognathism: aesthetic and functional results. Aesthetic Plast Surg 1998;22(3): 185±9.

[14] Ichiro O, Takehiko O, Eiji N, Kunihiro K, Isao M, Shinji N, Noboru O, Youichi U, Yoshiharu W, Fumiki T, Takeshi K. Three-dimensional analysis of craniofacial bones using three-dimensional computer tomography. J Craniomaxillofac Surg 1992;20:49±60.

[15] Rozema FR, Otten E, Bos RRM, Boering G, Willigen JDV. Computer-aided optimization of choice and positioning of bone plates and screws used for internal ®xation of mandibular fractures. Int J Oral Maxillofac Surg 1992;21(6):373±7.

[16] Svensson B, Feldmann G, Rindler A. Early surgical-orthodontic treatment of mandibular hypoplasia in juvenile chronic arthritis. J Craniomaxillofac Surg 1993;21(2):67±75.

[17] Finkelstein JA, Gross AE, Davis A. Varus osteotomy of the distal part of the femur: a survivorship analysis. J Bone Joint Surg Am 1996; 78(9):1348±52.

[18] James HB. Orthopaedic knowledge update. In: Howard SAn, Randall TL, editors. Spine, Rosemont: American Academy of Orthopaedic Surgeons, 1999. p. 561±754.

[19] Nagel A, Insall JN, Scuderi GR. Proximal tibial osteotomy. A subjective outcome study. J Bone Joint Surg Am 1996;78(9):1353±8. [20] Flynn JC, Hoque MA. Anterior fusion of the lumbar spine. End-result study with long-term follow-up. J Bone Joint Surg Am 1979;61(8): 1143±50.

[21] Gar®n SR, Vaccaro AR. Orthopaedic knowledge update. In: Gar®n SR, Vaccaro AR, editors. Spine, Rosemont: American Academy of Orthopaedic Surgeons, 1997. p. 55±61.

[22] Hall JE. Dwyer instrumentation in anterior fusion of the spine. J Bone Joint Surg Am 1981;63A:1188±90.

[23] Hodgson AR, Stock FE. Anterior spinal fusion: a preliminary communication on the radical treatment of Pott's disease and Pott's paraplegia. Br J Surg 1956;44:266±75.

[24] Muschik M, Zippel H, Perka C. Surgical management of severe spondylolisthesis in children and adolescents. Anterior fusion in situ

versus anterior spondylodesis with posterior transpedicular instru-mentation and reduction. Spine 1997;22(17):2036±42.

[25] Regan JJ, Yuan H, McAfee PC. Laparoscopic fusion of the lumbar spine: minimally invasive spine surgery. A prospective multicenter study evaluating open and laparoscopic lumbar fusion. Spine 1999;24(4):402±11.

[26] Chao EY, Barrance P, Genda E, Iwasaki N, Kato S, Faust A. Virtual reality (VR) techniques in orthopaedic research and practice. Stud Health Technol Inform 1997;39:107±14.

[27] Weingartner T, Dillmann R. Simulation of jaw-movements for the musculoskeletal diagnoses. Stud Health Technol Inform 1997;39: 401±10.

[28] Cutting C, Grayson B, McCarthy JG, Thorne C, Khorramabadi D, Haddad B, Taylor R. A virtual reality system for bone fragment positioning in multisegment craniofacial surgical procedures. Plast Reconstr Surg 1998;102(7):2436±43.

[29] Hsieh MS, Tsai MD. Diagnosis of herniated intervertebral disc assisted by three-dimensional, multiaxial, magnetic resonance imaging. J Formos Med Assoc 1999;98(5):347±55.

[30] Tsai MD, Hsieh MS. Surface rendering for multi-axial cross-sections. J Inform Sci Engng 2001;17(1):113±32.

[31] Tsai MD, Chang WC, Hsieh MS, Wang SK. Volume manipulation algorithms for simulating musculoskeletal surgery, Paci®c Graphics, 1996. Hsinchu, Taiwan: National Chiao Tung University, 1996 p. 220±34.

[32] Tsai MD, Hsieh MS, Chang WC. Distributed virtual environment for volume based surgical simulation via the world wide web, Proceed-ings of ICS'98 Workshop on Computer Graphics and Virtual Reality. Tainan, Taiwan: National Cheng Kung University, 1998 p. 7±14. [33] Feng L, Soon SH. An effective 3D seed ®ll algorithm. Comput Graph

1998;22(5):641±4.

Ming-Shium Hsieh received his MD degree in Medicine from Taipei Medical University, Taiwan, in 1974, and the PhD degree in Orthopedic Surgery from Essen University, Essen, Germany, in 1982. Since 1986, he has been the faculty of the Department of Orthopedics at Taipei Medical University, Taipei, Taiwan, where he is currently the chairman and an associate professor. His research interests include computerized graphics, image studies with clinical application, and orthopedic ®eld including spine surgery, arthroplasty and traumatology.

Ming-Dar Tsai received his BS degree in mechanical engineering from National Taiwan University, Taipei, Taiwan, in 1983, and MS and PhD degrees in machinery precision engineering from the University of Tokyo, Tokyo, Japan, in 1988 and 1991, respectively. Since 1991, he has been the faculty of the Department of Information and Computer Engineering, Chung Yuan Christian University, Chungli, Taiwan, where he is currently an associate professor. His research interests include computer graphics, virtual reality, scienti®c visualization and computer in medical applications.

Wen-Chien Chang received the DDS from Taipei Medical University, Taiwan, in 1980. Since 1989, he has been the faculty of the Oral and Maxillofacial Division, Dental Department at Taipei Medical Univer-sity, Taipei, Taiwan, where he has been the chairman of the department. His research interests include image studies and dental ®eld including oral and maxillofacial surgery, reconstruction surgery.