影像科學與數學(大學)

(1)

Imaging Sciences

and

Mathematics

I-Liang Chern Fall 2010

(2)

Imaging Sciences

• The SIAM Journal on Imaging Sciences covers all areas of imaging sciences, broadly

interpreted. It includes – image formation (imaging) – image processing

– image analysis

– image interpretation and understanding – computer graphics and visualization

– inverse problems in imaging;

• leading to applications to diverse areas in

science, medicine, engineering, and other fields.

(3)

Imaging Sciences

• Image Acquistion (Imaging)

- human vision, Optics, Radar imaging, Ultrasound, MRI, X-ray CT,…

• Image Processing

• Image Interpretation (Visual Intelligence)

] [ _input output T input I T I I ¾¾® =

(4)

Image Processing

Ø What is Image?

Ø What is Image Enhancement?

Ø Contrast Enhencement Ø Image Denoising Ø Image Deblurring Ø Image Inpainting Ø Image segmentation Ø Image Registration

Book: Rafael C. Gonzalez and Richard E. Woods,

(5)

What are Digital Images?

1. What is a digital image?

A digital image Is an array, or a matrix , of square pixels (picture elements) arranged in columns and rows.

a. Binary Image (logical array)

{ } ( , ) 1 0 I i j = or l k R n j m i I R I :W ® ¾sampling,¾¾¾quantized¾¾® _d :{1£ £ ,1£ £ }® _k,1£ £ Chiu-Yen Kao

(6)

What are Digital Images?

b. Intensity Image

8 bit (uint8, 0-255), 16 bit (uint16, 0-65535) and double ([0 1]) c. color Image

RGB:

24 bit = 256^3 ~ 16 million colors

(7)

What are Digital Images?

c. index color Image

data matrix and colormap matrix

(8)

Examples of images

• Daily-life images

• Astro images

(9)

(10)

(11)

(12)

Image Processing

(13)

Image Enhancement

1. Image Enhancement a. Intensity Adjustment b. Denoise c. Deblur Chiu-Yen Kao

(14)

Image Inpainting

“Image Inpainting : An Overview”, Guillermo Sapiro “Fast Digital Image Inpainting”, Manuel M. Oliveira, Brian Bowen, Richard McKenna and Yu-Sung Chang

(15)

Introduction to Image Segmentation

Chiu-Yen Kao _X _R _R _R _for _i _j

j i i N i ¹ = Ç = È = 0 , 1 Chiu-Yen Kao

(16)

Tumor(green), Vessels(red), Ventricles(blue), Edema (orange)

Image Registration

(17)

Image Processing

(18)

Contrast enhancement-1

(19)

Contrast enhancement-2

Histogram equalization

( , ) [ ( , )]

(20)

Contrast Enhancement -3

(21)

Image Processing

(22)

Noise models

• Assume white noise • Types of noises – Additive noise – Multiplicative noise – Mixed 2 , : mean 0, variance g = +f n n

s

2 , : mean 1, variance g = fn n

s

( , ) and ( ', ') are uncorrelated

n x y n x y

1 2

(23)

Noise Models-2

g = f + n

(24)

Noise Models-3

(25)

Denoise methods

• Filtering techniques

– Spatial filtering • Mean filters • Order-Statics filters – Frequency filtering – Wavelet filtering

• Variational approach

(26)

Spatial filtering

• Mean filters:

– Arithmetic mean filter

– Geometric mean filter

– Harmonic mean filter

, 1 ( , ) ( , ) ( , ) x y mn s t S f x y g s t Î é ù = ê ú ê ú ë

Õ

û ! g = +f n , ( , ) 1 ( , ) ( , ) x y s t S f x y g s t mn _Î =

å

! , ( , ) ( , ) 1 ( , ) x y s t S mn f x y g s t Î =

å

! g = fn

(27)

(28)

Mean filtering

• Convolution with a smoothing mask 1 2 1 2 4 2 1 2 1 1 16 , , , | |,| | 1 : i j s t i s j t s t f h g h g _- -£ = * =

å

! , s t h

(29)

Denoise methods

• Filtering techniques

• Variational approach

(30)

Original Image (a triangle)

Image corrupted by Impulse Noise

Only a number of pixels are corrupted

Impulse Noise Model

Noise

(31)

q Malfunctioning pixels in camera sensors

q Faulty memory locations in hardware

q Transmission in a noisy channel Two types of Impulse Noise

I. Salt-and-Pepper Noise

II. Uniformly-Distributed Random Noise Impulse Noise are caused by

(32)

(33)

Noise-free Image At 10% Noise

At 30% Noise At 50% Noise

(34)

(35)

Denoising Schemes

Median Filter

Sort Recovered

Noisy Image Restored Image

(36)

Median filter

(37)

q Drawback of Median Filter: Every pixel is modified, hence fuzziness and blurring

q Extensions of Median Filters (Median-type Filters):

q Adaptive Median Filter (Wang, IEEE

Trans IP, (1995))

q Adaptive Center Weighted Median Filter (2001)

q Multi-state Median Filters (2001)

q Filter based on homogeneity info (2003)

q …

q Detection statistics (IEEE TIP 2007)

(38)

Adaptive Median Filter

Sort

Noisy Image

(39)

Two Steps

1. Noise Detection (e.g., thresholding)

2. Noise Replacement (by Median or its variants)

Advantages

1. Fast

2. Accurate Detection

(40)

Median Filter

Adaptive Median Filter

(41)

Denoise methods

• Filtering techniques

– Spatial filtering • Mean filters • Order-Statics filters – Frequency filtering

• Variational approach

(42)

Frequency filter

• Noise in frequency

(43)

Frequency filtering

• Taking Fourier transform:

• Noise model:

• Band reject/pass filter

( ) ˆ ( , ) ( , ) i x y f x h =

òò

f x y e- x h+ dxdy

ˆ

_{( , )}

_{( , ) ( , )}

_ˆ

f

!

x h

=

k

x h

g

x h

ˆ

ˆg

= +

f

N

(44)

(45)

Denoise methods

• Filtering techniques

• Variational approach

(46)

Variation approach-1

• Noise model:

• Find a smooth solution under constraint

• If the solution is to minimize H1 norm

we call it H1 regularization

z u n

= +

u

2 2 |u z- | = s

ò

2 : mean 0, variance n s 2 | Ñu |

ò

(47)

Variational approach to denoising-2

• H1 denoising

• Total variation denosing

2 2 min |_u

ò

u z- | +a

ò

|Ñu |

(

) 0

u

u z

aD -

-

=

2 min |_u

ò

u z- | +a

ò

| Ñu | ( ) 0 | | u u z u

a

Ñ × æ_ç Ñ ö_÷ - - = Ñ è ø Euler-Lagrange equation Euler-Lagrange equation Regularization penalty

(48)

Why Total variation denoising

• TV norm: Keep edge sharp

Picture by Vogel and Oman Rudin, Osher, Fatemi

TV norm is insensitive to jumps (edges)

(49)

Image Processing

(50)

Blur model-1

• Convolution

• If h is a positive weight, then h*f is an averaging process, i.e. blurring

• Example: Finite size mask

( ) [ ]( ) : ( ) ( ) g x = *h f x =

ò

h x y f y dy -1 2 -1 2 4 2 1 2 1 , 1 16 s t h =

(51)

Blur model-2

2 2 ( )

ˆ( , )

k

h

x h

=

e

- x +h g h f= * 2 2 5 / 6 ( )

ˆ( , )

k

h

x h

=

e

- x h+ Atmospheric turbulence Gaussian model

ˆ ˆ

ˆg h f

= ×

(52)

(53)

Blur model -4

K : Translation 0 0 0 ( , ) T ( ( ), ( )) g x y =

ò

f x x t y y t dt-

(54)

-Blur model-5

g h f

= * +

n

h: Blur operator n: noise

(55)

Deblur methods

• Deconvolution in frequency domain

– Inverse filtering – Wiener filtering – …

• Deconvolution via wavelets

• Variational approach

(56)

Deconvolution

• Inverse filtering

• Wiener filtering

ˆ ˆ ˆ ˆ g h f= * + Þ = × +n g h f n 1 ˆ _ˆˆ _ˆ ˆ f kg g h = = ! 2 ˆ _{{ , }}ˆ ˆ ˆ ˆ ˆ ˆ _{ˆ ˆ} | | { , } { , } hE f f k h E f f E n n = + f! = *k g

(57)

Deblur-1

(58)

Deblur-2

(59)

Deblur methods

• Deconvolution in frequency domain

– Inverse filtering – Wiener filtering – …

• Deconvolution via wavelets

(60)

Debur via TV regularization-1

• Blur model

• Total variation regularization:

• Alternative formulation g h f= * + n 2 min_f a

ò

| Ñ +f |

ò

| h f* - g | 2 2 , min _{f w}a

ò

| |w +b

ò

| Ñ -f w | +

ò

| h f* - g | Y Wang et al.

(61)

Deblur via TV regularization-2

(62)

Deblur via TV regularization-3

(63)

Imaging Sciences

• Image Acquistion (Imaging)

- human vision, Optics, Radar imaging, Ultrasound, MRI, X-ray CT,…

• Image Processing

• Image Interpretation (Visual Intelligence)

] [ _input output T input I T I I ¾¾® =

(64)

What is imaging?

• Use physical methods to get geometrical

or physical properties of the objects

– Geometry: shape, morphology, structure,… – Physical properties:

• Mechanical: density, pressure, velocity,

concentration, viscosity, diffusion coefficients,… • Electrical: potential, current, impedance,

conductivity, resistance,

• Optical: absorption/reflection… • nuclear

(65)

Medical imaging

(Wiki)

• 1 Projection radiography

• 2 Tomography

• 3 Ultrasound

• 4 Fluoroscopy

• 5 Magnetic resonance imaging (MRI)

• 6 Nuclear medicine

• 7 Positron emission tomography (PET)

(66)

(67)

Tomography

Basic principle of tomography: superposition free tomographic cross sections S1 and S2 compared with the projected image P

(68)

Type of Tomography-1

• Atom probe tomography (APT) • Computed tomography (CT)

• Confocal laser scanning microscopy (LSCM) • Cryo-electron tomography (Cryo-ET)

• Electrical capacitance tomography (ECT) • Electrical resistivity tomography (ERT) • Electrical impedance tomography (EIT)

• Functional magnetic resonance imaging (fMRI) • Magnetic induction tomography (MIT)

• Magnetic resonance imaging (MRI), formerly known as

magnetic resonance tomography (MRT) or nuclear magnetic resonance tomography

(69)

Type of Tomography-2

• Optical coherence tomography (OCT) • Process tomography (PT)

• Positron emission tomography (PET)

• Positron emission tomography - computed tomography

(PET-CT)

• Quantum tomography

• Single photon emission computed tomography (SPECT) • Seismic tomography

• X-ray tomography (CT, CATScan)

• Photoacoustic tomography (PAT), also known as

Optoacoustic Tomography (OAT) or Thermoacoustic Tomography (TAT)

(70)

(71)

(72)

Nobel winners for CT (1979)

Allan McLeod Cormack Godfrey Hounsfield

(73)

Image Reconstruction

• Tomographic reconstruction

:

• Radon transform • Imaging model • Image reconstruction 1

( , )

( ) ,

x r

Rf

r

f x dx

S

q

× =

=

ò

Î

Given , reconstruct

z

u

z Ru n

=

+

(74)

(75)

(76)

(77)

(78)

Basic Principles of Nuclear

Magnetic Resonance

• Atoms with odd number of protons and/or neutrons possess nuclear spin angular

momentum S

• Associated with S is a magnetic dipole moment • Magnetic dipole moment rotates under external

magnetic field, exhibit magnetic resonance phenomena

• The variation of rotation of spins generates magnetic fluxes and can be recorded

• Hydrogen H+ atoms are abundant in biological specimens

(79)

MRI:

use magnetic fields to perform

•Relaxation: Main field B0

•Excitation: Radio Frequency (RF) field B1 •Fourier transform: Gradient field G

(80)

MRI is a Fourier integrator

• RF excitation selects a slice of magnetic dipoles

• The gradient field generates Fourier transform of magnetic dipoles Frequency encoding Phase encoding Gradient echo

(81)

Magnetic Resonance Imaging

Input Physical Agent Black Box Output Information Carrier Information Recording & Decoding EM waves Pulse sequences Transmit coils Contrast agents Magnetic dipoles of mobile protons

EM waves Receive coils

Image reconstruction Data processing Data analysis - T1 & T2 - Flow - Diffusion - Perfusion - Temperature - Cell tracking - Molecules

(82)

(83)

Summary of Imaging Sciences

• Imaging (data acquisition): CT, MRI

– Solving inverse problems

• Image processing:

– Enhancement (contrast enhancement, denoising, deblurring,…)

– Segmentation (edge detection, active contours,…)

(84)

Image science and mathematics

• Image science is important in

medicine

• Low dose, high resolution imaging

methods are needed

(85)