Jenho Tsao and Jia-Hong He
Dept. of EE. And Inst. of Comm., National Taiwan University, Taipei, Taiwan, ROC Abstract- The efficacy of Extracorporeal Shock Wave
Lithotripsy (ESWL) depends greatly on the capability to focus shock waves on renal stone. To achieve automatic focusing on moving target, the target must be under tracking. A mesh-based block matching algorithm is proposed for renal stone tracking using ultrasound image sequence. Since multiple targets are tracked together, the mesh-based tracking algorithm can provide a function of contextual regularization for solving the target missing and image degradation problems in renal stone tracking. Recorded ultrasound images of kidney during ESWL treatment are modified for demonstrating the capability of this algorithm.
I. INTRODUCTION
Extracorporeal Shock Wave Lithotripsy (ESWL) is the clinical procedure for disintegrating renal stones by shock waves from the outside of body [1]. The shock waves must be focused onto the stones. However, stones may move around during the course of treatment due to respiration, patient movement and shock-wave pressure. This will cause miss-hit of shock waves on surrounding tissues and cause trauma to tissues [2]. Respiratory triggering, general anesthesia, and real-time stone tracking [3] are means proposed to keep the stones inside the focal zone of shock wave. The real-time stone tracking approach was proven to be quite effective in reducing miss-hits and thus reduces tissue injury and treatment time [3, 4].
In the renal stone tracking system developed by Orkisz et al. [3], ultrasound image sequence are processed to provide positions of renal stones for real-time automatic shock-wave focusing. A simple block matching algorithm [5] is employed for image tracking. Each track is initialized manually by the physician and terminated when the cross correlation between successive image blocks is lower than a threshold. Tracking ultrasound images is difficult due to tissue deformation, noisy images, motion ambiguities, spatial aliasing, speckle decorrelation, out-of-plane motion, speckle motion artifacts and quantization error[6]. These artifacts make the target trajectory fluctuate fast, since the simple block matching is essentially a speckle tracking technique. The out-of-plane motion happens regularly due to respiration, which makes target disappear temporarily.
In addition to these problems, tracking stone under This work is supported by the National Science Council, Taiwan, ROC. (NSC95-2221-E002-442).
E-mail: tsaor215@cc.ee.ntu.edu.tw
fragmentation by ESWL is more challenging. Since after fragmentation, stone breaks to be small ones, which makes the stone image change all the time and its contrast degraded.
A mesh-based algorithm was proposed in [6] to cope with the speckle problems in tracking ultrasound image sequences. Unlike the simple block matching used in [3], which uses a single target block to track the stone, mesh-based algorithm uses more blocks to track the motion field over entire image plane. The embedded mesh puts a systematic control on the displacement estimations of the independent image blocks, which is known as mesh regularization.
For renal stone tracking problem, the scenario viewed by the ultrasound scanner is quite stationary over frames separated by 1/30 sec. The pressure caused by respiration makes the stone and its surrounding tissues move coherently to similar directions. Therefore, if more trackers were employed to track the stone and its surrounding tissues simultaneously, the mesh regularization scheme, which enforces the stone and tissues to move consistently, will improves the stability of the trajectory of stone.
It is found in this study that the mesh regularization scheme may provide more than motion regularization, it can provide contextual regularization also. When stone and tissues are tracked simultaneously, the existence of tissues (or background targets) will provide contextual information to infer the existence of stone (target). It is almost impossible that all background targets can disappear simultaneously, therefore, if most of the background targets were tracked properly, the mesh tracker will work properly, even the target disappears for a few frames. That is, the target tracker can be regularized by tissue trackers to tolerate the nonstationary degradation of target images. This helps in the relief of the out-of-plane motion problem and the target fragmentation problem. For the same situation, if there is only a single target tracker, there would be no way to prevent the tracker to switch its track to nearby tissue sites. Once a miss tracking happens, the tracker may keep wrong forever, even the target comes back latter.
The mesh-based tracking algorithm is given in section II.
Performances of this algorithm are given in section III. The contextual regularization capability is demonstrated by simulation studies in section III.
Proceedings of the 29th Annual International Conference of the IEEE EMBS
Cité Internationale, Lyon, France August 23-26, 2007.
FrP1B7.3
II. THE MESH-BASED TRACKING ALGORITHM
Since mesh-based tracking algorithms are well documented in literature, only a limited amount of description about the algorithm will be given below. For more details, readers may consult the works in [6-9]. The algorithm is a semi-automatic tracking algorithm. It needs manual initialization to supply the target and mesh related information, which will be described after the block matching criteria and mesh regularization are given. Some details about the mesh design will be given in the section of experimental test. After initialization, the algorithm does the node displacement estimation by block matching and mesh regularization on the node displacements repeatedly for successive frames.
A. Block Matching Criteria
For tracking target with speckled image, it was proposed in [3] to use the normalized texture correlation as the criteria for block matching. Let M be the set of pixel-positions within a matching block and the sampled data from the kth image frame be fk( );x x M∈ , the normalized texture correlation for forward matching is
1 standard deviation off xk( ),andd is the displacement vector to be estimated by maximizingC d( )within a limited search range. This is essentially a speckle tracking technique. It suffers from all speckle problems mentioned previously.
B. Mesh Regularization
A mesh can be considered as a connected spring system [8,9]. Given a set ofNnodes located at xn =( , )x yn n T and their connectivity, a mesh can be defined. Let
( , )
n = ∆xn ∆yn
u be the deformation (i.e., displacement) of thenthnode, all of the node deformations can be collected to be a deformation vector
1 1
( x, y, xn, yn, xN, yN)T
= ∆ ∆ "∆ ∆ "∆ ∆
U . Based on the finite
element theory, a stiffness matrix K can be found for e computing the deformation energy of the mesh as
( ) T
m e
J U = U K U .
Based on the theory of active mesh [8], the optimal estimate of the displacement vectorU should maximize the global (ensemble) correlation of all N blocks of image data
1
U u , as wells as the deformation energy of mesh model J Um( ) . Since J Uc( ) is mathematically untraceable, there is no efficient algorithm to find the
optimal solution. Yet, a suboptimal solution can still be constructed [6, 7], which is to estimate the independent displacements of each node first and then use them to find a mesh-regularized solution of U latter.
Let dn =(dx dyn, n)T be the displacement estimate of the nth node based on block matching, which maximizes
( )
C d already. For all mesh nodes, we have the observed n displacement vector D=(dx dy1, 1,"dx dyn, n,"dx dyN, N)T based on the image data. The mesh-regularized solution can be found by minimizing the sum of squared difference between U and D and the mesh deformation energy
T e
U K U. Using the Lagrange technique, the object function to be minimized is
2
where λ is a regularization parameter. This is essentially a constrained least square solution, which tries to minimize the discrepancy between the mesh-regulated displacement U and the observed node displacement D . If D is estimated by block matching, it will result in a mesh-regulated block matching algorithm for estimatingU . By differencing JCLS( )U , it is easy to show that the solution for minimizingJCLS( )U can be found by solving a matrix of equations and its solution is
( ) 1
CLS = +λ e −
U I K D ,
where I is an identity matrix of size 2N×2N. C. Algorithm Initialization
There are two major works to be done in algorithm initialization. They are the node allocation and the node labeling to specify the target and tissue nodes. Usually the mesh nodes are placed at positions of image feature points, such as edge points, corners, or points with high texture energy[6-9]. By so doing, it will result in an adaptive mesh, which has irregularly shaped mesh elements. Since the feature points change all the time, positions and labels of the mesh nodes must be updated for each frame and the stiffness matrix must be recomputed also. Furthermore, this will require a separate algorithm for target identification to keep the labels of each node.
For ESWL application, stone must be tracked in real-time, the use of adaptive mesh may cause computation problems.
Therefore, a fix-sized regular mesh with quadrilateral elements is adopted in this study. In this way, the mesh nodes will be allocated only once in the initialization step of the tracking algorithm. Thus the mesh model is time invariant, the regularization operator (I+λKe)−1need to be computed also once only.
III. RESULTS A. Experimental Test
An image sequence recorded during ESWL treatment is used for testing the proposed tracking algorithm. It contains sixty thousand frames of ultrasound images of a renal stone inside pelvis. The size of the stone is about 10 by 30 pixels (= 3.75 by 11.25 mm) as shown in Fig.1. This makes the choice of element size of mesh to be 11 by 11 pixels, then the target will have 3 nodes. To make the target nodes be surrounded by a ring of tissue nodes, the mesh must have at least 3 by 5 nodes. The image blocks used for displacement estimation is given in fig.1 and the mesh model of 15 nodes is given in fig.2. The nodes-7, 8 and 9 are target nodes. All 15 nodes are tracked simultaneously using the proposed mesh-based block matching algorithm. The stiffness matrix of the mesh is computed by the finite element method. The elasticity properties of the mesh model are set with Poisson’s ratio = 0.5 and Young’s modulus = 40kPa. The search range of block matching is ±4pixels.
The (x,y) positions of node-8 estimated by the mesh- based block matching algorithm are recorded as the trajectories of the stone, which are shown in Fig. 4 and 6.
For comparison, the (x,y) trajectories of tracking using the simple block matching are shown in fig. 3 and 5. As expected, the mesh regulation scheme makes the trajectories stable. For example, the marked fast movement at frame# 610 in fig.3 has been supressed, which is erroneous since it has a speed of 9cm/sec. The two x-trajectories has larger difference than the difference of the y-trajectories. This is because that the x-axis resolution is lower than the y-axis resolution, and the interference from the ESWL system is much serious in x direction.
0 200 400 600 800 1000
B. Simulation Study
To test the capability of the mesh regularization scheme, the displacement estimator at node 8 is interfered artificially by adding zero-mean Gaussian noise to the observed displacement
y8
d . The artificial interference has a standard deviation of 2 pixels and the modified observation is truncated to be within the search range of ±4 pixels, when the interference is too large. The artificial interference starts from frame-300 and ends at frame-500. The interfered displacement vector D is then used for solving the regularized displacement estimate UCLS . The modified observation
y8
d is shown in Fig.7. The mesh-regularized 14
Fig. 2 The mesh model used for displacement regularization Fig. 1 The image blocks used for displacement estimation,
the block size is 11 by 11 pixels
Fig. 3. The x-trajectory of renal stone using simple block matching.
Fig. 4 The x-trajectory of renal stone using mesh regularization.
Fig. 5 The y-trajectory of renal stone using simple block matching.
Fig. 6 The y-trajectory of renal stone using mesh regularization.
0 200 400 600 800 1000
u with regularization parameter λ=1 is shown in Fig.8. It is apparent that the extra error in
y8
d is suppressed to a compatible level as those frames without interference. The mesh regularized y-trajectory of renal stone is shown in Fig.9. It can be found that the interference does cause errors to the regularized trajectory estimate, however, comparing it to Fig.6, the stone is still under tracking properly. This result justifies that the mesh-regularized block matching algorithm may allow some node to be out-of-track through the function of contextual regularization.
For comparison, the mesh regularized displacement
y8
u with regularization parameterλ=2 is shown in Fig.10.
Compared it to Fig.9, it shows that the interference can be suppressed more by increasingλ . However, there is a trade-off between the discrepancy error U D and the − 2 degree of regularization.
IV. CONCLUSION
Mesh regularization can reduce the tracking error due to speckle. The mesh-based algorithm has the potential to provide contextual regularization for solving the target missing and image degradation problems in renal stone tracking. Through the use of the contextual information from tissue trackers, it might save the requirement of target recognition for ensuring that the ESWL device is shooting the stone instead of tissues.
ACKNOWLEDGMENT
Experimental data are supported from LiteMed Co. Taipei, Taiwan, ROC.
REFERENCES
[1] Chaussy, C., Schmiedt, E., Jocham, D. et al: First clinical experience with extracorporeally induced destruction of kidney stones by shock waves. J Urol, 127: 417, 1982
[2] Kuwahara, M.-A., Kambe, K. and Taguchi, K.: Initial experience using a new type extracorporeal lithotripter with an anti-misshot control device. J. Lithotripsy Stone Dis., 3:
141, 1991.
[3] M. Orkisz, M. Bourlion, G. Gimenez, T.A. Flam, “Real-time target tracking applied to improve fragmentation of renal stones in extra-corporeal lithotripsy,” Machine Vision and Applications, vol. 11, pp. 138–144, 1999.
[4] Chang, C. C., Liang, S. M., Pu, Y. R. et al: In vitro study of ultrasound-based real-time tracking for renal stones in shock wave lithotripsy: Part I. J Urol, 166: 28, 2001
[5] Jain, J. R. and Jain, A. K.: Displacement measurement and its application in interframe image coding. IEEE Transaction on Communication, COM, 29: 1799, 1981.
[6] F. Yeung, S. F. Levinson, D. Fu, and K. J. Parker,
“Feature-Adaptive Motion Tracking of Ultrasound Image Sequences Using A Deformable Mesh,” IEEE Trans. Medical Imaging, vol. 17, No. 6, pp. 945- 956, Dec. 1998
[7] S. Malassiotis and M. G. Strintzis, ”Tracking Textured Deformable Objects Using a Finite-Element Mesh,” IEEE Trans. Circuits Syst. Video Technol., vol. 8, pp. 756-774, Oct.
1998.
[8] Y. Wang and O. Lee, “Active mesh—A feature seeking and tracking image sequence representation scheme,” IEEE Trans.
Image Processing, vol. 3, pp. 610–624, Sept. 1994.
[9] D. Terzopoulos and M. Vasilescu, “Sampling and reconstruction using adaptive meshes,” in Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR’9I), June 1991, pp. 70-75.
Fig.7 The observed displacement
y8
d with artificial interference from frame-300 to 500.
Fig.8 The regularized displacement
y8
u with λ= . 1
Fig.9 The mesh regularized y-trajectory of renal stone with λ=1.
Fig.10 The mesh regularized y-trajectory of renal stone with λ=2.