Breast Ultrasound Imaging

1

Computer Aided Diagnosis in Breast Ultrasound Imaging

Ruey-Feng Chang (張瑞峰)

Department of Computer Science and Information Engineering (資訊系)

Graduate Institute of Biomedical Electronics and Bioinformatics (生醫電資所) National Taiwan University(國立台灣大學), Taiwan

Yi-Hong Chou

(周宜宏)

Veterans General Hospital, Taiwan

Chiun-Sheng Huang (黃俊升)

National Taiwan University Hospital, Taiwan

Yeun-Chung Chang (張允中)

National Taiwan University Hospital, Taiwan

Jeon-Hor Chen (陳中和)

University of California, Irvine, USA

Woo Kyung Moon

Seoul National University Hospital, Korea

Etsuo Takada

Dokkyo University, Japan

2

Contents

 2-D Breast US CADx

 Elastography CADx

 3-D Breast US/ABUS CADx

 ABUS CADe

 ABUS/MRI Density Analysis

 MRI CADe/CADx

3

NTU CAD Lab

 1997

Aloka SSD 1200

 2003

Acuson, GE, Medison, ATL, Philips

 1999

Free-hand 3D US

 2000 Voluson 530

 2002 GE 730

 2010 GE E8 and E6

2-D US 1997

3-D US

2000 2002

Whole Breast

 2002 Free-hand

 2003 Aloka ABUS

 2005 U-systems

2004

 2004

Telemed, Lithuania Echo Blaster 128

 2005

Terason t3000

PC-based US

Elastography

2006

MRI

2008 2011

Automated US

 2006

Hitachi Elastography

 2010

Siemens Elastography

 2011

Siemens ABVS

2011, 2013

2D/3D Shearwave

 2008

Morphology, Kinetic

 2009 Tofts Model

 2010 DWI

 2010 Siemens ARFI

 2011 Supersonic Shearwave

 20133D Elastography

Introduction

 For 2D breast US, the physician has detected the tumor and only the

computer-aided diagnosis (CADx) is needed for the tumor.

 For the new automated whole breast US (ABUS), the computer-aided

detection (CADe) is needed for detecting the tumors, just like mammography CAD.

4

Technology Development Program for Academic (TDPA, 學界科專)

 This TDPA project was supported by the Ministry of Economic Affairs

(MOEA) to develop

– a CADe system for ABUS – a CADx system for B-mode

US/elastography

– breast US GPS/recoding System

 The Co-PIs are Dr. Chou from VGH, Dr. Huang, and Dr. Chang from NTUH.

 8 PhD students worked on this

project.

Technology Development Program for Academic (TDPA, 學界科專)

 Two patents have been applied.

– Breast Ultrasound Scanning and Diagnosis Aid System

– Ultrasound Imaging Breast Tumor

Detection and Diagnostic System and Method

 25 international journals and 12 international conference papers

– Three IEEE Trans. MI papers have been published.

 The CADe and CADx systems have

been transferred to TaiHao Medical Inc.

(http://taihaomed.com/)

6

CADx for TDPA

8

CADx for TDPA

ABUS CADe

9

DEMO

10

DEMO

11

Breast US GPS/Recoding System

12 Tracking map Captured US video

Tumor location Probe direction

2-D BREAST US CAD

- ROBUST TEXTURE ANALYSIS

“Computer-aided Diagnosis Applied to US of Solid Breast Nodules by Using Neural Networks”, Radiology, vol. 213, no. 2, pp.407-412, 1999.

“Robust texture analysis using multi-resolution gray-scale invariant

features for breast sonographic tumor diagnosis,” IEEE Transactions on Medical Imaging, vol. 32, no. 12, pp. 2262-2273, 2013.

“Computer-aided diagnosis for distinguishing between triple-negative breast cancer and fibroadenomas based on ultrasound texture features,”

Medical Physics, vol. 42, no. 6, pp. 3024-3035, 2015.

13

14

2D US CAD

“Computer-aided Diagnosis Applied to US of Solid Breast Nodules by Using Neural Networks”, Radiology, vol. 213, no. 2, pp.407-412, 1999.

Editorial: Georgia D. Tourassi, "Journey toward Computer-aided Diagnosis: Role of Image Texture Analysis,” Radiology, vol. 213, no.2, pp.317-320, Nov. 1999.

Robust Texture Analysis

 Because the parameters of ultrasonic machine is adjustable, the images from the same machines may have different texture information.

 Moreover, the images from different ultrasonic machines have different texture information.

 Hence, the ranklet transform is

proposed in this study to extract the

gray-scale invariant texture features for tumor diagnosis.

“Robust texture analysis using multi-resolution gray-scale invariant features for breast 15

sonographic tumor diagnosis,” IEEE Transactions on Medical Imaging, vol. 32, no. 12, pp.

2262-2273, 2013.

Ranklet Transform

 Similar to wavelet transform, the ranklet images can be derived from a family of multi-resolution, orientation-selective features

 Differently from wavelet transform, it deals with ranks of pixels rather than with their gray-scale intensity values

16 HH (Diagonal subband, D)

HL (Horizontal subband, H) LH (Vertical subband, V)

Ranklet Example

 The ranklet images from images with non-linear monotonic change filters (i.e., gamma correlation, histogram equalization) are nearly the same.

17

Origin Wavelets Ranklets

W_-1^V W_-1^H W_-1^D R₈^V R₈^H R₈^D

W_-1^V W_-1^H W_-1^D R₈^V R₈^H R₈^D

Proposed Robust Texture Analysis

 First, we decompose each BUS image from the test database into ranklets.

 Afterwards, the gray-scale invariant texture features based on GLCM can be calculated from each

transformed image with regions of interest (ROIs).

 The SVM classifier is adopted to distinguish a benign tumor from a malignant one.

18

Input BUS Data GLCM Texture

Feature Extraction

Training Phase

Testing Phase

Ranklet Transform Ranklet Decomposition

I₀

H 1

R- R_-^V₁

D 1

R-

1

R-

Texture Feature Representation

SVM Training/Testing

Input Database SVM

Training/Testing

Three US Machines

 Database A includes 116 subjects (78 benign and 38 malignant cases) obtained with Acuson Sequoia

machine.

 Database B includes 193 subjects (133 benign and 60 malignant

cases) obtained with GE LOGIQ 7 machine.

 Database C includes 161 subjects (104 benign and 57 malignant

cases) obtained with GE Voluson 730 expert machine.

19

Experiments

 The GLCM-based textural features are applied for original image (origin), multi- resolution wavelet images (wavelets)

and multi-resolution ranklet images (ranklets).

20

D.B. Method AUC ACC (%) SENS (%) SPEC (%)

A

Origin ^{0.81±0.03 74.28±2.27 63.93±5.78 79.39±3.14} Wavelets ^{0.84±0.03 79.45±1.73 70.54±4.23 83.76±2.39} Ranklets ^{0.90±0.02 81.68±1.69 69.66±4.63 87.55±2.15}

B

Origin ^{0.86±0.03 78.75±2.21 66.53±5.99 84.33±2.89} Wavelets ^{0.92±0.02 84.23±1.71 74.99±4.74 88.38±2.12} Ranklets ^{0.94±0.02 86.35±1.64 79.56±4.44 89.35±2.11}

C

Origin ^{0.84±0.03 76.49±2.28 67.74±5.30 81.31±3.32} Wavelets ^{0.85±0.02 77.14±1.74 69.27±3.87 81.48±2.51} Ranklets ^{0.92±0.02 84.58±1.70 81.50±3.66 86.19±2.44}

Cross-machine Experiments

 The AUC of cross-platform training and testing

21

Train

Database Method Test Database

A

B C B+C

Origin (95% CI)

0.783 (0.717-0.846)

0.823*

(0.769-0.892) 0.786 (0.739-0.835) Wavelets

(95% CI)

0.722†

(0.6283-0.788) 0.752†

(0.663-0.816) 0.729†

(0.679-0.782) Ranklets

(95% CI)

0.934*

(0.896-0.965) 0.877

(0.825-0.929)

0.876*

(0.837-0.909)

B

A C A+C

Origin (95% CI)

0.724 (0.633-0.816)

0.807*

(0.736-0.869) 0.764†

(0.707-0.817) Wavelets

(95% CI)

0.757†

(0.643-0.828) 0.842*

(0.784-0.906) 0.832*

(0.779-0.879) Ranklets

(95% CI)

0.867 (0.792-0.929)

0.873 (0.817-0.922)

0.875*

(0.825-0.909)

C

A B A+B

Origin (95% CI)

0.709†

(0.614-0.806) 0.789

(0.720-0.860)

0.765†

(0.708-0.821) Wavelets

(95% CI)

0.795 (0.677-0.864)

0.785†

(0.712-0.854) 0.808†

(0.731-0.843) Ranklets

(95% CI)

0.859 (0.780-0.913)

0.913*

(0.871-0.949) 0.891*

(0.855-0.925)

TNBC vs. Fibroadenoma

 Triple-negative breast cancer (TNBC) is frequently misclassified as fibroadenoma due to benign

morphologic features on breast ultrasound.

 The multi-resolution ranklet features are proposed to to discriminate between TNBC and benign

fibroadenomas.

22

1cm benign fibroadenoma 1cm triple-negative breast cancer

“Computer-aided diagnosis for distinguishing between triple-negative breast cancer and fibroadenomas based on ultrasound texture features,” Medical Physics, vol. 42, no. 6, pp.

3024-3035, 2015.

Experiments

 169 tumors, including 84 benign fibroadenomas and 85 TNBCs, are used in this study

23

Accuracy (%) Sensitivity (%) Specificity (%) PPV (%) NPV (%) Az

Morphology 76.92^*

(130/169)

78.82^* (67/85)

75.00^* (63/84)

76.13^* (67/88)

77.78^*

(63/81) 0.8470^*

Conventional Texture (GLCM)

79.29^* (134/169)

81.18^* (69/85)

77.38^* (65/84)

78.41^* (69/88)

80.25^*

(65/81) 0.8542^*

Invariant Ranklet texture

87.57 (148/169)

89.41 (76/85)

85.71 (72/84)

86.36 (76/88)

88.89

(72/81) 0.9695

Combined 93.49

(158/169)

94.12 (80/85)

92.86 (78/84)

93.02 (80/86)

93.98

(78/83) 0.9702

ROC Curves

24

Elastography CADx

- SHEAR-WAVE ELASTOGRAPHY

“Analysis of elastographic and B-mode features at sonoelastography for breast tumor classification”, Ultrasound in Medicine and Biology, vol. 35, no. 11, pp. 1794–1802, 2009.

“Automatic selection of representative slice from cine-loops of real-time sonoelastography for classifying solid breast masses”, Ultrasound in Medicine and Biology, vol. 37, no. 5, pp. 709-718, 2011.

“Breast Tumor Classification Using Fuzzy Clustering for Breast

Elastography”, Ultrasound in Medicine and Biology, vol. 37, no. 5, pp.

700-708, 2011.

“Classification of breast tumors using elastographic and B-mode

features: Comparison of automatic selection of representative slice and physician-selected slice of images”, Ultrasound in Medicine and Biology, vol. 39, no. 7, pp. 1147-1157, 2013.

25

Dynamic Stain Elastography

 A lot of elastography slices are obtained during the operator to compress a tumor.

– That is, the stain elastography image is dynamic.

26

Benign Malignant

Elastography CAD

Benign or Malignant

Elastographyfeatures B-modefeatures

Selecting representative slice

Segmentation

 A representative slice of the dynamic elastography image needs to be

selected.

 The B-mode image could be used to segment the tumor and the tumor

contour could be applied to the corresponding

elastography image.

 Both the elastography features and B-mode

features could be used to diagnose the tumor.

“Analysis of elastographic and B-mode features at sonoelastography for breast tumor classification”, Ultrasound in Med. & Biol., Vol. 35, No. 11, pp. 1794–1802, 2009.

27

Image Quality Quantification

 Signal to Noise Ratio (SNR) and Contrast to Noise Ratio (CNR) are used to quantify the elastography image quality.

– The middle block in the tumor is used to calculate the SNR.

28

Low SNR High SNR

SNR mean

standard_deviation



“Automatic selection of representative slice from cine-loops of real-time sonoelastography for classifying solid breast masses”, Ultrasound in Med. & Biol., Vol. 37, No. 5, pp. 709-718, 2011.

Image Quality Quantification

 In the CNR method, not only the middle

block but also two outside blocks are used.

Low CNR High CNR

Over-compressed

2

2 2

( )

_

middle_mean outside_mean CNR middle_SD outside SD

 



The Strain Curve

30

SNR CNR

Tumor Segmentation

 The contrast-enhanced gradient

image is used to segment the tumor using the level set method.

Gradient Magnitude Sigmoid

Sigmoid Level Set

Result

31

BI-RADS Features

 Shape

– Oval, Round, Irregular

 Orientation

– Parallel, Not parallel

 Margin

– Circumscribed

– Not circumscribed

• Indistinct, Angular, Microlobulated, Spiculated

 Lesion boundary

– Abrupt interface, Echogenic halo

 Echo pattern

– Anechoic, Hyperechoic,

Complex, Hypoechoic, Isoechoic

 Posterior shadowing

– No posterior acoustic features, Enhancement, Shadowing,

Combined patter

BI-RADS Atlas

Elastography Features

 Stiffness Ratio

 Elasticity mean

 Lesion boundary elasticity

33

Experimental Results

 The data consisted of 151 biopsy-proved lesions (89 benign and 62 malignant lesions) from Dr. Moon.

 For CNR, elastography (82.12%) is better than B-mode (80.79%).

For SNR and Physician-selected, B-mode (87.42%, 84.11%) is better than elastography (82.12%, 82.78%).

 SNR (90.07%) is better than CNR (86.09%) .

SNR (90.07%) is similar to Physician-selected (89.40%) .

34

Features Accuracy Sensitivity Specificity PPV NPV

CNR

B-mode 80.79% 70.97% 87.64% 80.00% 81.25%

Elastography 82.12% 74.19% 87.64% 80.70% 82.98%

All Features 86.09% 82.26% 88.76% 83.61% 87.78%

SNR

B-mode 87.42% 83.87% 89.89% 85.25% 88.89%

Elastography 82.12% 79.03% 84.27% 77.78% 85.23%

All Features 90.07% 90.32% 89.89% 86.15% 93.02%

Physician selected

B-mode 84.11% 77.42% 88.76% 82.76% 84.95%

Elastography 82.78% 74.19% 88.76% 82.14% 83.16%

All Features 89.40% 85.48% 92.13% 88.33% 90.11%

“Classification of breast tumors using elastographic and B-mode features: Comparison of automatic selection of representative slice and physician-selected slice of images”, Ultrasound in Medicine and Biology, vol. 39, no. 7, pp. 1147-1157,2013

ROC curves for SNR Representative Slice

Elastography, Az=0.8461 B-mode, Az=0.9223

B-mode + Elastography Az=0.9407

35

True Negative Case

CNR SNR

Physician-selected

Fibroadenomas

True Positive Case

CNR SNR

Physician-selected

Infiltrating carcinomas

Summary

 The performance of B-mode features is better than that of elastography features in this study.

 Combining the B-mode and

elastography features could improve the diagnosis performance.

 The proposed image quantification methods could have similar

performances with the physician.

– The proposed selection of representative slice is robust and reliable.

38

Shear-wave Elastography

 Stain elastography requires manual tissue compression which is operator dependent.

 Shear-wave elastography (SWE) is a new method to permit absolute quantification of tissue stiffness.

 Using acoustic radiation to stimulate tissues

39

Benign Softer (low kPa)

Malignant Harder (high kPa)

Shear-wave CAD

40

Shear wave

image Segmentation

B-mode features

Elastographic features Seed

Diagnosis

Result

Experimental Results

 SuperSonic Imagine Aixplorer US

 109 breast tumors

– 57 benign and 52 malignant

41

109 cases: 57 benign and 52 malignant

Accuracy (%) Sensitivity (%) Specificity (%) PPV (%) NPV (%) Az

B-mode 87.16

(95)

86.54 (45)

87.72 (50)

86.54 (45/52)

87.72

(50/57) 0.8312

Elastography 89.91 (98)

86.54 (45)

92.98 (53)

91.84 (45/49)

88.33

(53/60) 0.9511

Combined 94.50

(103)

92.31 (48)

96.49 (55)

96.00 (48/50)

93.22

(55/59) 0.9705

ROC Curves

42

3D/4D Elastography

 The new CAD systems could be further developed for the 3D/4D elastography to provide more robust diagnosis.

43

From Dr. Takada From Dr. Moon and Dr. Chou

3-D BREAST US/ABUS CADX

“Characterization of Spiculation on Ultrasound Lesions”, IEEE Transactions on Medical Imaging, vol. 23, no. 1, pp. 111-121, 2004.

“Solid Breast Masses: Neural Network Analysis of 3-D Power Doppler Ultrasound Image Features for Classification as Benign or Malignant”, Radiology, vol. 243, no. 1, pp. 56-62, 2007.

“Analysis of Tumor Vascularity Using Three-Dimensional Power Doppler

Ultrasound Images,” IEEE Transactions on Medical Imaging, vol. 27, no. 3, pp.

320-330, 2008.

“Vascular Morphology and Tortuosity Analysis of Breast Tumor Inside and Outside Contour by 3-D Power Doppler Ultrasound”, Ultrasound in Med. &

Biol., vol. 38, no. 11, pp. 1859-1869, 2012.

“Computer-aided diagnosis for the classification of breast masses in automated whole breast ultrasound images,” Ultrasound in Medicine and Biology, vol. 37, no. 4, pp. 539-548, 2011.

44

45

3D US Spiculation

 A kind of stellate distortion caused by the intrusion of the breast cancer.

 The spiculation displayed only on the C- view of 3D US.

– The conventional 2D US cannot see the spiculation.

*IEEE Transactions on Medical Imaging, vol. 23, no. 1, pp. 111-121, Jan. 2004. ⁴⁵

Tumor

Spiculation

46

Experimental Result

 Spiculations are more easily seen in malignant tumors.

 Spiculations are more likely to appear in the coronal view.

0 40 80

1 120

(a) Detection slices

Dataset no.

0 40 80

1 120

(b) Detection slices

Dataset no.

0 40 80

1 120

(c) Detection slices

Dataset no.

Longitudinal view

0 40 80

1 105

(b) Detected slices

Dataset no.

0 40 80

1 105

(c) Detected slices

Dataset no.

0 40 80

1 105

(a) Detected slices

Dataset no.

Coronal view Transverse view

120 benign 3D datasets

105 malignant 3D datasets

3D Power Doppler US

 A useful tool to detect the vessels.

– 2D Doppler US can not obtain the entire 3-D vessels.

 The blood supply and vessel distribution are the important features to diagnose an tumor.

“Solid Breast Masses: Neural Network Analysis of 3-D Power Doppler Ultrasound Image 47

Features for Classification as Benign or Malignant”, Radiology, vol. 243, no. 1, pp. 56-62, 2007.

48

System Overview

Preprocessing

Feature Extraction

Neural Network 3-D

Thinning Algorithm Building

Vascular Tree Original Dataset

Malignancy Prediction vascular points

one-pixel wide skeleton

Thinning Result

Vascular Tree Construction

 To generate a skeleton representation, the thinning result was converted into tree structure using breadth first search (BFS) algorithm.

Bifurcate node root

x

y

z

●

●

●

●

○

○ ○ ○

○

○

○ ○ ○

root

x

y

z

●

●

●

●

○

○ ○ ○

○

○

○ ○ ○

51

Feature Extraction

 Ten features are used for the vessel tree.

– Vessel-to-Volume Ratio (R

)

– Number of Vascular Trees (N

) – Measurement of Length

• Total length (L

)

• Length of the longest path (L

)

– Bifurcation (Bn) – Diameter (D

)

– Number of cycles (NC)

– Tortuosity measures (DM, ICM, SOAM)

52

Benign Case

Original Thinning

All Vessels

Longest

Vessel

53

Malignant Case

Original Thinning

All Vessels

Longest

Vessel

54

Benign vs. Malignant

 The bottom images are the longest vessel.

 The longest vessel of malignance has more curvature.

Benign Malignant

55

Vascular Analysis Inside/outside Tumor

 To evaluate morphologic and tortuous

features of vessels inside and outside the tumor region on 3-D power Doppler US

“Vascular Morphology and Tortuosity Analysis of Breast Tumor Inside and Outside Contour 55

by 3-D Power Doppler Ultrasound”, Ultrasound in Med. & Biol., vol. 38, no. 11, pp. 1859-1869, 2012.

Benign

56

Malignant

The significant vessel trees outside (red) and inside (white) tumor.

The tumor contour (blue) and primary path outside tumor (red) and inside tumor (white)

2-D PDUS Vessels (orange) and skeletons

(red and white)

ROC Curves

 There were 113 solid breast masses (60 benign, 53 malignant).

57

ABUS CADx

 2D/3D CADx could be applied for ABUS.

58 3-D breast tumor VOI

Level set segmentation

B-mode image Tumor mask

Texture feature Shape feature Ellipsoid fitting feature

Classification using binary logistic regression

“Computer-aided diagnosis for the classification of breast masses in automated whole breast ultrasound images,” Ultrasound in Medicine and Biology, vol. 37, no. 4, pp. 539-548, 2011.

3D Tumor Segmentation

Benign

Malignant Example

60

 147 cases (76 benign and 71

malignant breast masses) were obtained by U-systems ABUS.

61 GLCM Shape Ellipsoid

fitting

GLCM and shape

GLCM and ellipsoid

fitting

Shape and ellipsoid

fitting

ALL

Accuracy (%) _(111/147)^75.51 _(121/147)^82.31 _(117/147)^79.59 _(118/147)^80.27 _(120/147)^81.63 _(125/147)^85.03 _(121/147)^82.31 Sensitivity (%) _(58/71)^81.69 _(60/71)^84.51 _(54/71)^76.06 _(59/71)^83.10 _(60/71)^84.51 _(60/71)^84.51 _(59/71)^83.10 Specificity (%) _(53/76)^69.74 _(61/76)^80.26 _(63/76)^82.89 _(59/76)^77.63 _(60/76)^78.95 _(65/76)^85.53 _(62/76)^81.58 PPV (%) _(58/81)^71.60 _(60/75)^80.00 _(54/67)^80.60 _(59/76)^77.63 _(60/76)^78.95 _(60/71)^84.51 _(59/73)^80.82 NPV (%) _(53/66)^80.30 _(61/72)^84.72 _(63/80)^78.75 _(59/71)^83.10 _(60/71)^84.51 _(65/76)^85.53 _(62/74)^83.78

Az ^{0.8603 0.9138 0.8496 0.9153 0.8195 0.9466 0.9388}

ABVS CADe

- TUMOR DETECTION - TUMOR MAPPING

“Computer-Aided Tumor Detection Based on Multi-Scale Blob

Detection Algorithm in Automated Breast Ultrasound Images”, IEEE Transactions on Medical Imaging, vol. 32, no. 7, pp. 1191-1200, July 2013.

“Tumor detection in automated breast ultrasound images using quantitative tissue clustering”, Medical Physics, vol. 41, no. 4, pp.

042901-1-8, April 2014.

“Multi-dimensional tumor detection in automated whole breast ultrasound using topographic watershed,” IEEE Transactions on Medical Imaging, vol. 33, no. 7, pp. 1503-1511, July 2014.

“Feasibility Testing: Three-dimensional Tumor Mapping in Different Orientations of Automated Breast Ultrasound,” Ultrasound in

Medicine and Biology, vol. 42, no.5, pp. 1201-1210, 2016. ₆₂

Automated Whole Breast US

63

 Aloka, U-systems (acquired by GE), and Siemens, SonoCiné (distribution agreement with Philips), and iVu have developed the automated whole breast ultrasound (ABUS) systems.

Aloka, 2006.07

Aloka

U-systems

U-systems, 2006.09

Siemens

SonoCiné

iVu

Siemens ABVS

 ACUSON S2000 Automated Breast Volume Scanner (ABVS)

 Siemens: 736×481×318 = 0.208mm×0.052mm×0.526mm

ABVS Viewer

 Our ABVS view system has been

transferred to TaiHao Medical Inc, Taiwan.

– BR-ABVS Viewer 1.0

– FDA cleared, 2016.01

– http://taihaomed.com/

ABVS Viewer (2)

 Our ABVS view system is applied for a ABUS system from a Taiwan company.

– The probe is rotated to obtain a whole breast image in only one scanning.

– Begin clinical trials in NTUH and other hospitals

 Cooperation with China company for CFDA

66

Supine/Prone ABUS

 Supine ABUS

– Same position with surgery

– At least three passes for a breast

 Prone ABUS

– Same position with MRI – One pass for a breast

 Reconstruction of rotated prone ABUS (iABUS/iVu)

67 Inferior

Lateral

AP Medial

Superior

iABUS Viewer

68

2.9 clock / 34 mm (123.9.91) Transverse view

87 degree Radial view Sagittal view

Coronal view

Dual-modality ABUS System

 Combines an FFDM X-ray machine with automated breast ultrasound (ABUS) technology

– CapeRay Medical Ltd.

– A long probe is scanned under the plate.

69

ABUS/MAMMO Viewer

70

Mammo

Watershed Tumor Detection

 Our ABUS CADe is based on a watershed transform method.

– The watershed transform was applied to

gather similar tissues around local minima to be homogeneous regions.

 This method detects every tumor, but

some non-tumors (false positive, FP) are also detected.

– Hence, the likelihoods of being tumors of the regions were estimated using the quantitative morphology, intensity, and texture features in the 2-D/3-D false positive reduction (FPR).

“Multi-dimensional tumor detection in automated whole breast ultrasound using topographic 71

watershed,” IEEE Transactions on Medical Imaging, vol. 33, no. 7, pp. 1503-1511, July 2014.

Proposed System

72

ABUS image

Resolution reduction &

image enhancement Watershed transform for tissue segmentation

Suspicious abnormality extraction

2-D/3-D false positive reduction

Slice by slice processing

False Positive Reduction (FPR)

2013/7/27

73 Original

image

Watershed transform

Abnormality extraction

2-D FPR

3-D FPR

Experiments

 ABUS SomoVu ScanStation

– 138 cases (104 abnormal and 34 normal) – 104 breast lesions of 104 patients

• 68 benign lesions

• 65 malignant lesions

• Lesion size : 1.75±1.13 cm

 10-fold cross validation (10F-CV) is used.

 FROC curves and the jackknife alternative of FROC-1 (JAFROC) figure of merit (FOM)

were performed to evaluate the performance of the CADe system

74

FPs/pass after applying 2-D/3-D FPR

 Before FPR, the average number of suspicious abnormalities was 291.6.

75

Sensitivity (%)

FPs/pass

After 2-D FPR After 3-D FPR

60 1.48 1.58

70 2.32 2.14

80 4.64 3.33

90 9.31 5.42

100 18.19 9.44

Reduce 94% Reduce 48%

FROC Curves

 The FROC curves and the

corresponding JAFROC-1 figure of merit (FOM) for each feature set.

76

9.44 FPs/pass

Detected Results

 A true positive case of 1.51 cm fibroadenomas

77

The potential tumor regions delineated

by watershed segmentation.

CAD The original ABUS image

Detected Results

 A true positive case of 4.46 cm ductal carcinoma in situ

78

The potential tumor regions delineated

by watershed segmentation.

CAD

The dot circle indicates the false

positive.

The original ABUS image

Detected Results

 A false positive case

79

The potential tumor regions delineated

by watershed segmentation.

CAD

The dot circle indicates the false

positive.

The original ABUS image

Discussion

 We also applied the multi-scale Hessian analysis (published in IEEE TMI) to the same dataset used in this study to

provide a performance comparison.

– “Computer-Aided Tumor Detection Based on Multi-Scale Blob Detection Algorithm in Automated Breast Ultrasound Images,”

IEEE Transactions on Medical Imaging, vol. 32, no.7, pp.1191- 1200, 2013.

 The proposed watershed method has better performance and short running time.

80

Detection rates 100% 90% 70% Running time

Hessian 18.0 9.1 4.3 13 minutes

Watershed 9.4 5.4 2.1 74.3 seconds

Summary

 A fully automatic CADe system was proposed based on topographic

watershed for analyzing ABUS image.

 The proposed CADe system showed sensitivities of 100% with 9.44

FPs/pass.

 Rib and shadow regions were misclassified

– Further reduce the FPs.

81

Tumor Mapping

 For ABUS, there are three views for a breast and a tumor will be

demonstrated in multiple views.

82

Experiment

 A total of 53 abnormal passes with 41 biopsy-proven tumors and 13 normal passes were collected.

 After CAD detection, a mapping pair was composed of a detected region in one pass and another region in another pass.

– Location criteria, including the radial position, relative distance and distance to nipple, were used to extract mapping pairs with close regions.

 Quantitative intensity, morphology, texture and

location features were then combined in a classifier for further classification.

 The performance of the classifier achieved a mapping rate of 80.39% (41/51), with an error rate of 5.97%

(4/67).

“Feasibility Testing: Three-dimensional Tumor Mapping in Different Orientations of 83

Automated Breast Ultrasound,” Ultrasound in Medicine and Biology, vol. 42, no.5, pp. 1201- 1210, 2016.

Mapping Case

84

L MED L AP

Benign fibroadenoma

ABUS/Mammo Mapping

85

Preliminary Clinical Result

 Three readers: radiologist, surgeon, sonographer

– 168 ABUS views from 28 patients

– With CADe, more tumors could be found

 Study design

– Step 1: Reader reviews the cases without CAD – Step 2: Reader reviews the missed CAD

markers at step 1

• Only the CAD marked missed at step 1 will be presented to reader.

• The missed CAD markers are decided by the distance of their nearest markers at step 1.

86

Step 1

Review without CAD

Step 2

Review the missed

CAD markers

Results

87

CADe Reader A Reader A

with CAD Reader B Reader B

with CAD Reader C Reader C with CAD

#TP 101 99 103 77 79 113 115

#FP 968 186 203 60 62 237 257

#FN 14 16 12 38 36 2 0

SEN 87.83 86.09 89.57 66.96 68.69 98.26 100

FP per

Pass 5.76 1.11 1.21 0.36 0.37 1.41 1.53

 The missed CAD markers and their

BI-RADS scores were re-evaluated by reader at step 2.

 Reviewing times of step 1 and step 2

88

Time per

View Step 1 Step 2

(CAD) Total CAD

Markers Time per Marker

Reader A 57.7 34.1 91.8 6.0 5.7

Reader B 26.0 12.7 38.7 6.1 2.1

Reader C 162.4 38.1 200.5 5.7 6.4

BI-RADS 3 4a 4b 4c 5 Total

Reader A 7 2 3 2 1 15

Reader B 2 1 0 0 1 4

Reader C 23 0 0 0 0 23

Case #1

 Reader B

– Not marked at step 1 but marked at step 2 (CAD) – BI-RADS 4a

 Readers A and C – Marked at step 1

89

Case #2

 Reader B

– Not marked at step 1 but marked at step 2 (CAD) – BI-RADS 3

 Readers A and C

– Not marked at step 1

90

Case #3

 Reader B

– Not marked at step 1 but marked at step 2 (CAD) – BI-RADS 3

 Readers A and C – Marked at step 1

91

Case #4

 Readers A and B

– Not marked at step 1 but marked at step 2 (CAD) – BI-RADS 5

 Reader C

– Marked at step 1

92

Free-hand Whole Breast US

 Using magnetic tracker and image capturing, the conventional US can

be used to scan the whole breast, like ABUS.

93 trakSTAR

unit

Transmitte r

Frame Grabber Prob

e

Sensor

Magnetic Trackers

94

Transmitter Electronics unit

Sensor

Hitachi

Sensor

Free-hand WBUS System

 The free-hand GPS system has been tried in the NTUH Breast Center and NTUH Yun-Lin Branch.

– The free-hand system will be compared with ABUS.

 In the viewing system, the user can select any location to reviewing the images at that location.

95 Free-hand GPS Image Recorder Free-hand GPS Image Player

ABVS/MRI/TOMO Density Analysis

- RIB SHADOW FOR FINDING BREAST REGION

“Breast density analysis for whole breast ultrasound images”, Medical Physics, Vol. 36, No. 11, pp. 4933-4943, Nov. 2009.

“Comparative study of density analysis using automated whole breast ultrasound and MRI”, Medical Physics, Vol. 38, No. 1, pp. 382-389, Jan. 2011.

“Breast Density Analysis with Automated Whole-Breast Ultrasound: Comparison with 3-D Magnetic Resonance

Imaging,” Ultrasound in Med. & Biol., vol. 42, no.5, pp. 1211- 1220, 2016.

96

97

Aloka Whole Breast Density Analysis

 Breast density measured from

mammograms and whole breast US

images Aloka whole breast US

Mammo

“Breast density analysis for whole breast ultrasound images”, Medical Physics, Vol. 36, No. 11, pp. 4933-4943, Nov. 2009.

98

Whole Breast US vs. Mammogram

Grade 3

Grade 4

Grade 2

U-systems Whole Breast Density Analysis

 The similar Aloka whole breast

density analysis could be applied for the U-systems.

99

Original C View

Gland/Fat Analysis

U-Systems Density Analysis

 The U-systems ABUS density and volume is highly correlated with those of MRI.

100

Density Volume

“Comparative study of density analysis using automated whole breast ultrasound and MRI”, Medical Physics, Vol. 38, No. 1, pp. 382-389, Jan. 2011.

MRI ABUS

FCM Density Analysis

 The fuzzy c-mean (FCM) classifier was used to differentiate the fibroglandular and fatty tissues in ABUS and MRI images

101

Original

FCM

U-Systems Density Analysis

 The U-systems ABUS density and volume is highly correlated with those of MRI.

102

Density Volume

“Comparative study of density analysis using automated whole breast ultrasound and MRI”, Medical Physics, Vol. 38, No. 1, pp. 382-389, Jan. 2011.

MRI ABUS

Rib Shadow for Finding Breast Region

 For density analysis, an automatic breast segmentation method was

proposed based on the rib shadow to extract breast region from ABUS.

103

Transverse View

Sagittal View

An Example

 Using rib shadow to extract the chest wall line.

104

Chest Wall Line

Segmented Breast Region

“Breast Density Analysis with Automated Whole-Breast Ultrasound: Comparison with 3-D Magnetic Resonance Imaging,” Ultrasound in Med. & Biol., vol. 42, no.5, pp. 1211-1220, 2016.

Experimental Results

 MRI and ABVS images of 46 breasts from 23 women were collected.

 Our results revealed a high correlation in WBV and BPD between MRI and ABVS.

 Our study suggests that ABVS provides breast density information useful in the assessment of breast health.

105

TOMO Density

106

0.2012

0.3363 0.2110

MRI TOMO 2D Mammo

BI-RADS=3

From Dr. Moon, SNUH

LCC

TOMO Density

107

0.3685 0.2106 0.2318

MRI TOMO 2D Mammo

BI-RADS=4

From Dr. Moon, SNUH

LCC

Mammo Volumetric Density

MRI Density

MammoVolumetric Density

MRI CADe/CADx

“Computerized breast lesions detection using kinetic and morphologic analysis for dynamic contrast-enhanced MRI,”

Magnetic Resonance Imaging, vol. 32, no. 5, pp. 514-522, 2014.

“Computerized breast mass detection using multi-scale Hessian- based analysis for dynamic contrast-enhanced MRI,” Journal of Digital Imaging, vol. 27, no. 5, pp. 649-660, 2014.

“Computer-aided diagnosis of mass-like lesion in breast MRI:

Differential analysis of the 3-D morphology between benign and malignant tumors,” Computer Methods and Programs in

Biomedicine, vol. 112, no. 3, pp. 508-517, 2013.

“Computer-aided diagnosis of breast DCE-MRI using

pharmacokinetic model and 3-D morphology analysis,” Magnetic Resonance Imaging, vol. 32, no. 3, pp. 197-205, 2014.

109

Hessian-based Detection

 Nonparametric nonuniform

intensity normalization algorithm (N4ITK) is used to reduce

nonuniformity bias.

 The deformation of thorax to

approximate the outline between breast and chest by the Demons deformable algorithm is used to segment the breast region.

 Fuzzy c-means algorithm (FCM) and Hessian algorithm are used to detect the tumors.

110

Bias field correction

Breast DCE-MRI

Motion correction

True tumor selection

Detected tumor Beast region segmentation

Tumor candidate detectionFCM

Hessian Preprocessing

Detection Result

111

Detection performance analysis

 True tumor detection performance

– The mass detection rates are 100% (61/61)

with 15.15 false positives per case and 91.80%

(56/61) with 4.56 false positives per case.

112

Breast DCE-MRI CADx

 The Pharmacokinetic Model and 3-D morphology Analysis are used to

diagnose the tumors in DCE-MRI.

113

3D breast DCE-MR images Tumor segmentation

Texture Feature Shape Feature Kinetic curve analysis

Tumor classification