TECHNICAL COMMUNICATION
A real time collaboration system for teleradiology
consultation
Jiann-Shu Lee
a,*
, Ching-Tsorng Tsai
b, Chen-Hsing Pen
c, Hui-Chieh Lu
daDepartment of Aeronautical Engineering, National Huwei Institute of Technology, 64 Wen-Hua Road, Huwei Jen, Yunlin, Taiwan 632, ROC
bDepartment of Computer andInformation Science, Tung-Hai University, Taichung, Taiwan cComputer andCommunication Center, Taichung Veterans General Hospital, Taichung, Taiwan
dDepartment of Computer andInformation Science, National Chiao Tung University, Hsinchu, Taiwan, ROC
Received 9 September 2002 ;received in revised form 7 July 2003;accepted 16 July 2003
KEYWORDS Collaboration; Teleradiology; PACS; Internet; Java
Summary Real time collaboration systems, in which participants share multimedia
data and applications in real time, have attracted many researchers in recent years. A teleradiology consultation system based on the real time collaboration technol-ogy is presented in this paper. Under the platform-independence consideration, Java technologies are employed to construct the system. Applying this system, an off-duty on-call radiologist can make diagnoses and report easily by viewing the transferred images at home. Owing to the accessibility of image, all users can examine and ma-nipulate images consistently such that a secluded hospital can be assisted to hold remote consultation. To reduce the network transmission time, the command-passing and local command execution techniques are utilized to achieve the screen synchro-nization. A pointer function is also developed to maintain the cursor consistency in a more efficient manner during consultation when a detail indication of the examined image is needed. Besides, a dialog window is also designed for on-line conversation. Since Java programs can run on heterogeneous platforms, the need for system main-tenance and user training can be substantially reduced.
© 2003 Elsevier Ireland All rights reserved.
1. Introduction
Collaboration has become practical because of the multidimensional communications usually employ both real time and non-real time tools. We can now access multiple modes data, which consists of text, files and multimedia, through communica-tion. The primary facilitator of collaborative work is telecommunications. Without the telecommuni-cations technology[1], the enhancement of produc-tivity and the amplification of creaproduc-tivity become
*Corresponding author.
virtually impossible. Knowledge workers will need computing and communicating tools to assist them with collaborative work. The requirement is espe-cially urgent in the health sciences domain, where work is characterized by geographic dispersion of co-workers and compressed time frame for work output[2—5].
Medical image play an important role in diagno-sis and treatment in the modern medicine system
[6,7]. The need of domain knowledge in analyzing the medical image urges the clinicians to consult with radiologists for further diagnosis. Teleradiol-ogy was introduced in 1972 [8] to solve the prob-lem of consultation, especially in remote areas and 1386-5056/$ — see front matter © 2003 Elsevier Ireland All rights reserved.
on a day-to-day basis. The number of private users accessing the Internet from their home is rising very quickly. For convenience, we utilize the Internet as our WAN instead of proprietary networks.
In this paper, a teleradiology consultation system is proposed by employing the real time collabora-tion technology. The underlying collaboracollabora-tion envi-ronment is based on the Transmission Control Pro-tocol/Internet Protocol (TCP/IP). The existing plat-forms are diverse. For hardware, the personal com-puter (PC) level equipments include Pentium and Macintosh and the popular operating systems such as Unix, Mac OS and Microsoft Windows. Although these platforms can be connected by network, spe-cial versions of programs still have to be designed for different platforms. Furthermore, the software development for the individual platform to com-municate with each other will take great effort. Java is a new programming language developed by Sun Microsystems for the World Wide Web[11]. Introduction of Java as a platform-independence programming language for the Internet is one of the most significant software breakthroughs for the Internet-related technologies. A Java program is compiled into bytecodes and class libraries in a standard form which can be supported by most of the existent popular platforms. Many Java run-time environments have been provided on the Internet, for example the Microsoft IE and the Netscape Com-municator have been approved of Java compliance. Using Java we can establish a teleradiology system which is independent of the underlying platforms. Besides, Java can dramatically reduce the cost of software distribution and maintenance. A Java gram can be executed by using either a local
pro-2. Methods
2.1. System design
There are several important aspects in design-ing a teleradiology system. Teleradiology, in the most general sense, means the transmission of ra-diographic images from one place to another. It may be further supported by video conferencing (telemedicine) so that the medical personnel at each end of the link can discuss with each other and annotate on the examined image. Thus, a so-phisticated teleradiology system should provide the synchronous consultation, the connection with the PACS for accessing images, the basic image ma-nipulation and processing functions, and the tele-conferencing for interactive conversation[12,13].
The developed system is particularly useful in providing immediate diagnosis for a remote hos-pital and in assisting imminent radiological report and consultation in urgent conditions when a doc-tor is at home. The system provides remote consul-tation in such way that both clinicians can review the same images and discuss with each other by the dialog window or microphone. To hold the re-mote consultation the screen synchronization is needed. If the screen content of one end is updated then the screen content on the other end will be refreshed simultaneously. To let the remote user can easily follow what you are concerned about, a specific indicator as shown in Fig. 1 is designed for the consistency indication to the interesting image area. In addition, a user can annotate by us-ing text, line segments, rectangles, or polygons on
Fig. 1 A displayed MR knee image during consultation. The synchronous cursor or pointer is pointing out the patella.
An annotation is indicated in the image. The connected site is exhibited by the IP address shown in the left bottom of the window.
the interesting areas of the examined image. And, these annotations can be almost simultaneously displayed on both ends as shown inFig. 1. A dialog window is also designed for an alternative on-line conversation as shown inFig. 2.
Fig. 3 depicts our system architecture. The sys-tem consists of three components: the image server,
Fig. 2 The interactive dialog box is presented on the consistent MR image during consultation.
the database (DB) server, and the client group-ware. The DB server and the image server are both connected to the Internet/Intranet. And, clients are directly synchronized with each other through the Internet/Intranet. The image server is used to store images. The DB server provides the patient’s demographic data and the corresponding images
Fig. 3 The composition of our system.
locations in the image server. The client groupware enables users to retrieve and manipulate images, and hold a remote consultation.
The client groupware is composed of a viewer program, a stream listener and a local image file. The viewer program requests the DB server and the image server to retrieve images and provides a graphic user interface (GUI) to process the ac-quired images. The local image file is used to store the retrieved images. The stream listener provides two-way on-line communications and is executed to maintain the consistency between both ends.
The DB server provides the location of the re-quired image to the client. Once the server is ac-tivated, it passively waits for a request from the client. When the DB server receives a request, it will query the patient DB for the corresponding
de-mographic data and reply the image location to the client. The client will then request the image server to transfer the desired images by file transfer pro-tocol (FTP).
2.2. System operation
Users operate the system through the client group-ware. When a client activates, the stream listener is triggered at the same time to wait for the con-nection from other remote clients. However, if the user specifies an IP address it will actively connect to the specified remote client. After connection, a synchronizing channel will be established between clients. Users can query the patient information from the DB server and retrieve the desired im-ages from the image server. These operations are
instantaneously informed to the remote clients by passing commands through the synchronous channel. The remote clients immediately execute the received commands to carry out the same operations.
The system can be used for both image consulta-tion and image reporting. We have designed the sys-tem as an alternative in remote PACS. If a radiolo-gist wants to report his diagnosis of images at home, he can also initiate the process without IP specifi-cation, which enables a radiologist to pre-fetch the required images before reporting.
2.3. System implementation
There is no client—server concept between the con-nected clients. The concon-nected client subsystems are symmetric. Any client can actively connect to or passively be connected by other clients. Every operation on one client will be instantaneously re-peated on the others by passing the same command. The symmetric property simplifies the user opera-tion and the implementaopera-tion complexity.
For consultation convenience, the screen con-tent synchronization is needed. There are two synchronization methods for screen display, which are screen-sharing strategy and command-passing strategy. The screen-sharing strategy is used by Intel ProShare and is commonly applied in telecon-ferencing, which can transmit the whole updated screen to others even though there are only few changes. Since any change on one end will cause a screen dump to the others, the network trans-mission load is heavy. For example, if we want to enhance the examined image by adjusting its window-level, the screen-sharing strategy will send almost a whole screen of data to other sides. This will induce roughly a million bytes of data to be sent on the network. However, in our system, we adopt the command-passing method, which only transmits a command code and parameters of the operation to the synchronized clients. It takes only a few bytes to represent the command code and pa-rameters. Having received the command code, the remote computers will execute this command and obtain the same screen display immediately. The command-passing strategy tremendously reduces the volume of transmission data and the response time for synchronization display. All operations, in-cluding image manipulation, annotation, and dialog are implemented by this strategy in our system.
However, synchronization display has some draw-backs, such as operation conflict and insignificant cursor movement. Operation conflict happens when more than one user activates the operations simul-taneously. This will result in an inconsistent screen
display. This conflict is solved by a token-passing strategy, which means that if a user wants to acti-vate a command he should press an icon first to get the control token.
Movement of a mouse cursor results in a string of insignificant events, and this will increase the volume of message transmission. This problem is solved by introducing a synchronized indica-tor, in which users move the indicator to point out the interesting area for discussion by the command-passing technique. Thus we need not transmit all movement events of mouse cursors.
Our system is implemented using JDK l.1.4 (Java Development Kit), and can be executed on JRE l.1.4 (Java Run-time Environment) or later versions. The Java applets for Web browsers such as Microsoft IE (Internet Explorer) or Netscape Communicator were not adopted in the system, because our program has to access local files and download images into the local disks. These actions violate the security constraints of Java applets.
3. Results
This system provides for consultation by viewing the radiological images directly. The acquired im-ages can be browsed on a side panel. The inter-ested image is selected by clicking the mouse and then the image is displayed in the main window. The examined image can be manipulated by the fol-lowing operations: brightness and contrast adjust-ment, window level and width settleadjust-ment, inver-sion, zooming in/out and measurement. A user can utilize a synchronized indicator to locate the inter-esting anatomy or lesion, which can also be anno-tated. A dialog window can be opened for interac-tive communication.
Our experimental system was established for the radiologist emergency consultation at home. By using a telephone line modem, a doctor can access the medical images from the PACS of the Taichung Veterans General Hospital and communicate with the consulting clinicians. Although this teleradi-ology system is convenient, the bandwidth of the channel becomes a bottleneck due to the massive data of medical images. To reduce the transmission time, a lossless compression scheme for medical images was applied, which is supported by the standard of Digital Imaging and Communications in Medicine (DICOM). The DICOM standard provides the mechanism for the use of JPEG compression through the encapsulated format. By this scheme, we can compress the file size of a medical image to a half or even to a quarter size of the origi-nal one. For example, an X-ray image with size of
ous image operations on a local computer is within 1 s, while it expends a few seconds for the remote computer to repeat the same operation through the command-passing technique.
4. Discussion
The greatest advantage of this system is that it provides a platform-independent teleradiology consultation service through the Internet. Re-mote consultations can be accomplished easily and cost-effectively. However, the bottleneck coming from the Internet bandwidth still exists. In order to speed up the image retrieval, we are looking for a faster transmission technique. An alterna-tive in solving this problem is to apply the pre-fect function of our system to retrieve images in advance.
For on-line consultation, the necessary equip-ment for the system is network interface or modem which is connected with the Internet. The system can employ a microphone and a sound card for audio conversation to replace telephone dialog. Because not all computers are equipped with an audio device, the dialog window is used as an al-ternative in consultation assistance. Furthermore, for teleconferencing video cameras are needed. In fact, there is no natural way to implement a video conversation environment in Java now. But a new feature of the Java platform, called the Java Media Framework (JMF), is going to be integrated into our system in the near future. It will become a great support to the media capture and conferencing function of this system.
data encryption problem. The encryption compat-ibility between hospitals should follow standards. We have not implemented the encryption mecha-nism in our system yet. However, the encryption implementation following the standard proposed by DICOM will be included in our system in the future. The main shortcoming of a Java system is that the system response seems too slow. Since some instructions supported by Java are not native to the platforms, e.g. Windows NT/95 or Unix, this makes the instruction cannot be executed in the most efficient way. Besides, the Java interpreter runs slowly as well. However, the utilization of the Java just-in-time compiler or the Java chips will im-prove the performance. In addition, the frequent upgrades of Java means that most programmers are not familiar with the newest version and most Java run-time systems do not support all of the newest features. The newly released Java class libraries or run-time environments may also be incompatible with the Chinese versions of operating systems. In order to avoid these problems, we should choose the class libraries carefully and test them under dif-ferent run-time environments in advance. However, this would increase the maintenance cost for our system.
5. Conclusions
The Internet has become a very popular and conve-nient telecommunications infrastructure for infor-mation exchange in recent years. More and more tools, including both real time and non-real time, have been developed to resolve the communication
problems based on the Internet. The great progress in the Internet has made remote collaboration become possible. This function is especially ur-gent in the health sciences domain because their work is characterized by geographic dispersion of co-workers and increasingly compressed time frame for work output. In this paper, we propose a teleradiology consultation system applying the real time collaboration technique on the Internet. Based on the platform-independent consideration, Java technologies were employed to design our system. Applying this system, the radiologist can provide consultation at home in emergency. In addition, this system can assist a rural hospital without a full-time radiologist with radiology reporting and remote consultation. All users can retrieve and manipulate images and hold remote consultation. To reduce the network transmission time, the command-passing and local command execution strategy was utilized to achieve the screen synchronization. In addition, to maintain the cursor consistency during consulta-tion in a more efficient way, i.e. when a detail indi-cation of the examined image is needed, a pointer function was developed. A dialog window was also designed for an alternative on-line conversation. Since Java programs can run on distinct platforms, the need for system maintenance and user training can be substantially reduced.
Acknowledgements
This report was partly supported by a grant from the Joint research program of Veterans General Hospital and National Tsing-Hua University in Taipei (VGHTH 85-024-3), sponsored by the Medical Re-search Advancement Foundation in memory of Dr. Chi-Shuen Tsou.
References
[1] A. Sykes, M. Symods, D. Van Doren, Collaboration in the Information Age, Business Communications Review, BCR Enterprises, Hinsdale, IL, May 1997, pp. 3—5.
[2] W.K. Wong, H.K. Huang, R.L. Arenson, J.K. Lee, Digital Archive System for Radiologic Images, InfoRAD 14, 1991 pp. 1119—1125.
[3] G. Irie, K. Miyasaka, HU-PACS. 10 months experience, Com-put. Methods Programs Biomed. 36 (1991) 71—75. [4] H. Mosser, M. Urban, W. Hruby, Filmless digital
radiology-feasibility and 20 months experience in clinical routine, Med. Inform. 19 (1994) 149—159.
[5] G. Svahn, S. Holtas, E.M. Larsson, E. Bengtsson, PACS for radiology conference—improvement of application soft-ware, Comput. Methods Programs Biomed. 43 (1994) 81— 84.
[6] M. Hosono, Y. Nakano, S. Urayama, J. Konishi, K. Uokawa, Y. Tanaka, Three-dimensional display of cardiac structures using reconstructed magnetic resonance imaging, J. Digital Imaging 8 (1995) 105—115.
[7] C.R. Guiado, P. Matinez, R. Roig, F. Mirosa, J. Salmeron, F. Florensa, M. Roger, Y. Barragan, Three-dimensional MR of the inner ear with steady-state free precession, Am. J. Neuroradiol. 16 (1995) 1909—1913.
[8] W.D. Bidgood, E.V. Staab, Understanding and using tel-eradiology, Semen Ultrasound CT MR 13 (1992) 102— 112.
[9] C.W. Yang, P.C. Chung, S.K. Lee, et al., An image capture and communication system for emergency computed to-mography, Comput. Methods Programs Biomed. 52 (1996) 139—145.
[10] T.C. Wu, S.K. Lee, C.H. Wen, et al., An economical PC-based picture archiving and communication system, Ra-diology 205 (1997) 746.
[11] J.S. Gage, An introduction to Java, MD Comput. 13 (1996) 476—480.
[12] H.K. Huang, Picture Archiving and Communication Systems in Biomedical Imaging, VCH, New York, 1996, pp. 329— 332.
[13] G.T. Barnes, R.L. Morin, E.V. Staab, Teleradiology: funda-mental considerations and clinical applications, in: J.C. Honeyman, E.V. Staab (Eds.), Syllabus: A Special Course in Computers for Clinical Practice and Education in Radiol-ogy, RSNA, Oak Brook, 1992, pp. 139—146.
