Adaptive Digital Zoom Adaptive Digital Zoom Techniques Based on Hypothesized Boundary

(1)

Adaptive Digital Zoom Adaptive Digital Zoom Techniques Based on Hypothesized Boundary

Author: 藍寅峻 (Y. C. Lan)

Written by: Ting-Hsuan Chang Mobile Phone: 09633-31533 E-Mail: r95922102@ntu.edu.tw

(2)

Outline Outline

z

Introduction

z

Weighting-Based Digital Zoom

z

Weighting Based Digital Zoom Algorithms

A B d R t ti d R l

z

Area-Based Restoration and Resample

z

Experimental Resultspe e ta esu ts

z

Conclusion

(3)

Outline Outline

z Introduction

z

A B d R t ti d R l

z

Conclusion

(4)

Introduction Introduction

z

Digital zoom is the process to scale up a digital image to another higher-resolution g g g image by using a computer.

z

We can observe details in an image by

z

We can observe details in an image by applying digital zoom algorithms.

(5)

Introduction (cont.) Introduction (cont.)

z

The major problem of digital zoom technique is that we only have little q y

information to generate a high-resolution image from a low-resolution one

image from a low resolution one.

z

Many researches have focused on super-resolution algorithms using multiple images.p g

(6)

Introduction (cont.) Introduction (cont.)

Th l ti t h i d l

z

The super-resolution technique deals

with images containing stationary scene with objects and captured by a moving camera.

z

However, we do not always have many images of stationary scene with objects.ages o stat o a y sce e t objects

z

The digital zoom approaches using a

single image are developed to solve this single image are developed to solve this difficulty.

(7)

Introduction (cont.) Introduction (cont.)

z

Because of the lack of information the intensity value of interpolated pixel is y p p guessed or interpolated by its

neighboring pixels neighboring pixels.

z

We can handle the problems in

ffrequency domain or in spatial domain.

(8)

Introduction (cont.)

(9)

Introduction (cont.)

(10)

Introduction (cont.) Introduction (cont.)

z

Left: Original image (Digital zooming in red rectangle)g )

z

Top right: nearest neighbor pixel copy B tt i ht bili i t l ti

z

Bottom right: bilinear interpolation

(11)

Introduction (cont.) Introduction (cont.)

z

Because the blurry and blocky effects appear on the edges when applying pp g pp y g bilinear interpolation.

z

We propose “adaptive digital zoom

z

We propose adaptive digital zoom techniques based on hypothesized

” ff

boundary” to deal with the effects in this thesis.

(12)

Outline Outline

z

Introduction

z Weighting-Based Digital Zoom

z Weighting Based Digital Zoom Algorithms

A B d R t ti d R l

z

Conclusion

(13)

Introduction Introduction

z

Our motivation is to keep the object

edges sharp and to have better results.g p

(14)

(15)

Terminologies and Algorithm

O i

Overview

z Object: An image is composed of many

objects. An object in an image is defined j j g as a region where the pixels in it have

similar property (such as intensity) similar property (such as intensity).

z Run: A run is defined as a segment of

pixels in an image scan line and with similar property.p p y

(16)

Terminologies and Algorithm

O i ( t )

Overview (cont.)

z Run boundary: The boundary of two

different runs in the same image scan g line is called run boundary.

z Hypothesized boundary (HB): The

hypothesized boundaries are obviously f

located from the run boundaries in the nearest scan line to the nearest ones in the third scan line.

(17)

Terminologies and Algorithm

O i ( t )

Overview (cont.)

z HB pixel set: A pixel set that hypothesized

boundary passes through.

(18)

Algorithm Overview Algorithm Overview

z We divide the scale up process into two sub-

processes, one for vertical and the other one

for horizontal process

(19)

Algorithm Overview (cont.) Algorithm Overview (cont.)

z

In the beginning of the adaptive

algorithm, we copy the intensity values of g , py y pixels in the original image to the

corresponding pixels in the scale-up corresponding pixels in the scale up image.

z

How to generate a pixel to be

interpolated in our algorithm depends on p g p whether the pixel is in HB pixel set or not.

(20)

Algorithm Overview (cont.) Algorithm Overview (cont.)

If it di t l f it i t l ti

z

If its gradient values of its interpolating pixels is larger than a user-defined

threshold, the pixel falls in an HB pixel set.

z

We use our weighting-based algorithm to deal with these kinds of pixels because dea t t ese ds o p e s because the hypothesized boundary passes

through it.

z

If not, we use the linear interpolation.

(21)

Algorithm Overview (cont.)

(22)

Linear Weighted Sum Algorithm Linear Weighted Sum Algorithm

NRB: Nearest Run-Boundary LRB: Left Run-Boundary RRB: Right Run Boundary RRB: Right Run-Boundary D: Distance

(23)

Linear Weighted Sum Algorithm ( t )

(cont.)

(24)

Linear Weighted Sum Algorithm ( t )

(cont.)

(25)

Nearest neighborhood i l

pixel copy

1x 2x

4x

(26)

Bilinear interpolation

1x 2x

4x

(27)

Linear weighted sum

1x 2x

4x

(28)

Sigmoid Weighted Sum Algorithm

(29)

Sigmoid Weighted Sum Algorithm ( t )

(cont.)

), ( )))

( )

( (

1 ( ) ( ))

( )

( (

)

(P₁_.₅ Sig D P₁ D P₂ I P₁ Sig D P₁ D P₂ I P₂

I = _NRB − _NRB × + − _NRB − _NRB ×

(30)

Sigmoid Weighted Sum Algorithm ( t )

(cont.)

(31)

C = 0.01

1x 2x

4x

(32)

C = 1.0

1x 2x

4x

(33)

Realization for a Binary Image:

L NRBD S l ti

Larger-NRBD Selection

NRBD: Nearest Run Boundary Distance

(a) Nearest-neighbor pixel copy

(b) Bilinear interpolation with binarization

(c) Linear weighted sum

(d) Linear weighted sum with binarization

(34)

Realization for a Binary Image:

L NRBD S l ti ( t ) Larger-NRBD Selection (cont.)

z

We propose a one-pass algorithm to replace the two-pass algorithm linear p p g weighted sum followed by binarization.

z

We may call the process as larger

z

We may call the process as larger- NRBD selection algorithm.

(35)

Realization for a Binary Image:

L NRBD S l ti ( t )

Larger-NRBD Selection (cont.)

(36)

Realization for a Binary Image:

L NRBD S l ti ( t ) Larger-NRBD Selection (cont.)

))), (

), ( max(

(arg )

(P₁_.₅ I D P₁ D P₂

I = _NRBD _NRBD

(37)

Take a Break

(38)

Outline Outline

z

Introduction

z

A B d R t ti d R l

z Area-Based Restoration and Resample

z

Conclusion

(39)

Introduction Introduction

z

We assume an intensity value of a pixel is the integration of the light energy in a g g gy Charge-Coupled Device (CCD) grid.

z

Based on the CCD grid mode and the

z

Based on the CCD grid mode and the hypothesized boundary concept we can

f

restore the infinite-high-resolution or continuous signal locally.g y

(40)

Introduction (cont.)

(41)

CCD Grid Mode and the Hypothesized

B d L li ti

Boundary Localization

z

We want to scale up the image two-by- two. We may divide a pixel in a low-y p

resolution image into four pixels to get a high-resolution image

high resolution image.

z

The pixel center in the low-resolution

image locates on one pixel center in the high-resolution image.g g

(42)

CCD Grid Mode and the Hypothesized

B d L li ti

Boundary Localization

(43)

CCD Grid Mode and the

H th i d B d L li ti

Hypothesized Boundary Localization

(a) Area-based algorithm.

(b) Linear weighted sum

( ) g

algorithm.

(44)

Local Restoration Local Restoration

(d), (e), (f) : mirrors of (a), (b), (c)

(45)

Local Restoration (cont.) Local Restoration (cont.)

z

Use (a) for an example. To count the area of A

₁

2 2

2 1

1 1

1

) (

) 1

( )

(

R

R R

I P

I

I A

P I

=

× +

×

−

= ¹

_I

R1: Original intensity of P₁ I_R2: Original intensity of P₂

2 2

)

(

_R

z

Let the area of each pixel be 1

R2 g y ₂

) 1

( )

( ₁ ₁ ₂

1 A

I A P

I_R I ^R

−

×

= −

et t e a ea o eac p e be

) (

) 1

(

2 2

1

P I I

A

R =

(46)

Local Resampling for Scaling up by T

Two

z

Using the high-resolution area divided by the hypothesized boundary to calculate yp y the intensity

z

Each high resolution pixel’s area

z

Each high-resolution pixel s area becomes 0.5

z

Take (a) for example

(47)

Local Resampling for Scaling up by T ( t )

Two (cont.)

' '

'

) ( 0 5 )

( P A I A I

I

2 '

5 . 1

2 1

1 1

1

) (

) 5

. 0 ( )

(

R

R R

I P

I

I A

P I

=

× +

×

−

=

2 '

2

)

( P I

_R

I =

(48)

Area-based algorithmg

1x 2x

4x

(49)

Outline Outline

z

Introduction

z

A B d R t ti d R l

z

z Experimental Results pe e ta esu ts

z

Conclusion

(50)

Experimental Results Experimental Results

z Visualization results

z Digital zoom on red g rectangle

z Five methods will be Five methods will be

compared.

(51)

Experimental Results (cont.) Experimental Results (cont.)

(a) Nearest-neighbor pixel copy (b) Bili i t l ti

(b) Bilinear interpolation (c) Linear weighted sum (d) Sigmoid weighted sum (d) Sigmoid weighted sum (e) Area-based algorithm

(52)

Experimental Results (cont.) Experimental Results (cont.)

(a) Original image

( ) g g

(b) Nearest-neighbor pixel copy

(c) Bilinear interpolation (d) Larger-NRBD

selection

(53)

Experimental Results (cont.) Experimental Results (cont.)

(a) Original image

( ) g g

selection

(54)

Experimental Results (cont.) Experimental Results (cont.)

(a) Original image

( ) g g

selection

(55)

SNR and Sharpness Comparison SNR and Sharpness Comparison

Original Image 4x4 nearest neighbor pixel copy Original Image 4x4 nearest neighbor pixel copy

Sharpness: 153 037 221 SNR: 9 551811 Sharpness: 153,037,221 SNR: 9.551811

Sharpness: 99,597,786 SNR: Signal-to-Noise Ratio

(56)

SNR and Sharpness Comparison ( t )

(cont.)

4x4 bilinear interpolation 4x4 linear weighted sum 4x4 bilinear interpolation 4x4 linear weighted sum

SNR: 12 807445 SNR: 12 730128

SNR: 12.807445

Sharpness: 23,033,292

SNR: 12.730128

(57)

SNR and Sharpness Comparison ( t )

(cont.)

4x4 sigmoid weighted sum 4x4 area-based algorithm 4x4 sigmoid weighted sum 4x4 area based algorithm

SNR: 12 655672 SNR: 13 046390

SNR: 12.655672

SNR: 13.046390

(58)

Outline Outline

z

Introduction

z

A B d R t ti d R l

z

z Conclusion

(59)

Conclusion Conclusion

z

Nearest-neighbor pixel copy

z

Advantage: fast, simple g , p

z

Disadvantage: blocky, useless in scaling up process

process

z

Bilinear interpolation

z

Advantage: fast, simple

z

Disadvantage: blocky and blurry effects on g y y

edges

(60)

Conclusion (cont.) Conclusion (cont.)

z Linear weighted sum

z

Advantage: efficient, fast, no blocky effects on edges, good visualization

z

Disadvantage: not sharp enough on edges

z Sigmoid weighted sum

z

Advantage: a user-defined parameter for tuning, sharpness larger than bilinear interpolation even linear weighted sum

z

Disadvantage: SNR smaller than bilinear

interpolation

(61)

Conclusion (cont.) Conclusion (cont.)

z

Larger-NRBD selection

z

Advantage: suitable for binary images g y g

z

Disadvantage: only for binary images

A b d l ith

z

Area-based algorithm

z

Advantage: good SNR and sharpness values

z