Adaptive Digital Zoom Adaptive Digital Zoom Techniques Based on Hypothesized Boundary
Author: 藍寅峻 (Y. C. Lan)
Written by: Ting-Hsuan Chang Mobile Phone: 09633-31533 E-Mail: r95922102@ntu.edu.tw
Outline Outline
z
Introductionz
Weighting-Based Digital Zoomz
Weighting Based Digital Zoom AlgorithmsA B d R t ti d R l
z
Area-Based Restoration and Resamplez
Experimental Resultspe e ta esu tsz
ConclusionOutline Outline
z Introduction
z
Weighting-Based Digital Zoomz
Weighting Based Digital Zoom AlgorithmsA B d R t ti d R l
z
Area-Based Restoration and Resamplez
Experimental Resultspe e ta esu tsz
ConclusionIntroduction Introduction
z
Digital zoom is the process to scale up a digital image to another higher-resolution g g g image by using a computer.z
We can observe details in an image byz
We can observe details in an image by applying digital zoom algorithms.Introduction (cont.) Introduction (cont.)
z
The major problem of digital zoom technique is that we only have little q yinformation to generate a high-resolution image from a low-resolution one
image from a low resolution one.
z
Many researches have focused on super-resolution algorithms using multiple images.p gIntroduction (cont.) Introduction (cont.)
Th l ti t h i d l
z
The super-resolution technique dealswith images containing stationary scene with objects and captured by a moving camera.
z
However, we do not always have many images of stationary scene with objects.ages o stat o a y sce e t objectsz
The digital zoom approaches using asingle image are developed to solve this single image are developed to solve this difficulty.
Introduction (cont.) Introduction (cont.)
z
Because of the lack of information the intensity value of interpolated pixel is y p p guessed or interpolated by itsneighboring pixels neighboring pixels.
z
We can handle the problems inffrequency domain or in spatial domain.
Introduction (cont.)
Introduction (cont.)
Introduction (cont.)
Introduction (cont.)
Introduction (cont.) Introduction (cont.)
z
Left: Original image (Digital zooming in red rectangle)g )z
Top right: nearest neighbor pixel copy B tt i ht bili i t l tiz
Bottom right: bilinear interpolationIntroduction (cont.) Introduction (cont.)
z
Because the blurry and blocky effects appear on the edges when applying pp g pp y g bilinear interpolation.z
We propose “adaptive digital zoomz
We propose adaptive digital zoom techniques based on hypothesized” ff
boundary” to deal with the effects in this thesis.
Outline Outline
z
Introductionz Weighting-Based Digital Zoom
z Weighting Based Digital Zoom Algorithms
A B d R t ti d R l
z
Area-Based Restoration and Resamplez
Experimental Resultspe e ta esu tsz
ConclusionIntroduction Introduction
z
Our motivation is to keep the objectedges sharp and to have better results.g p
Terminologies and Algorithm
O i
Overview
z Object: An image is composed of many
objects. An object in an image is defined j j g as a region where the pixels in it havesimilar property (such as intensity) similar property (such as intensity).
z Run: A run is defined as a segment of
pixels in an image scan line and with similar property.p p yTerminologies and Algorithm
O i ( t )
Overview (cont.)
z Run boundary: The boundary of two
different runs in the same image scan g line is called run boundary.z Hypothesized boundary (HB): The
z Hypothesized boundary (HB): The
hypothesized boundaries are obviously f
located from the run boundaries in the nearest scan line to the nearest ones in the third scan line.
Terminologies and Algorithm
O i ( t )
Overview (cont.)
z HB pixel set: A pixel set that hypothesized
boundary passes through.
Algorithm Overview Algorithm Overview
z We divide the scale up process into two sub-
processes, one for vertical and the other one
for horizontal process
Algorithm Overview (cont.) Algorithm Overview (cont.)
z
In the beginning of the adaptivealgorithm, we copy the intensity values of g , py y pixels in the original image to the
corresponding pixels in the scale-up corresponding pixels in the scale up image.
z
How to generate a pixel to beinterpolated in our algorithm depends on p g p whether the pixel is in HB pixel set or not.
Algorithm Overview (cont.) Algorithm Overview (cont.)
If it di t l f it i t l ti
z
If its gradient values of its interpolating pixels is larger than a user-definedthreshold, the pixel falls in an HB pixel set.
z
We use our weighting-based algorithm to deal with these kinds of pixels because dea t t ese ds o p e s because the hypothesized boundary passesthrough it.
through it.
z
If not, we use the linear interpolation.Algorithm Overview (cont.)
Algorithm Overview (cont.)
Linear Weighted Sum Algorithm Linear Weighted Sum Algorithm
NRB: Nearest Run-Boundary LRB: Left Run-Boundary RRB: Right Run Boundary RRB: Right Run-Boundary D: Distance
Linear Weighted Sum Algorithm ( t )
(cont.)
Linear Weighted Sum Algorithm ( t )
(cont.)
Nearest neighborhood i l
pixel copy
1x 2x
4x
Bilinear interpolation
1x 2x
4x
Linear weighted sum
1x 2x
4x
Sigmoid Weighted Sum Algorithm
Sigmoid Weighted Sum Algorithm
Sigmoid Weighted Sum Algorithm ( t )
(cont.)
), ( )))
( )
( (
1 ( ) ( ))
( )
( (
)
(P1.5 Sig D P1 D P2 I P1 Sig D P1 D P2 I P2
I = NRB − NRB × + − NRB − NRB ×
Sigmoid Weighted Sum Algorithm ( t )
(cont.)
C = 0.01
1x 2x
4x
C = 1.0
1x 2x
4x
Realization for a Binary Image:
L NRBD S l ti
Larger-NRBD Selection
NRBD: Nearest Run Boundary Distance
(a) Nearest-neighbor pixel copy
(b) Bilinear interpolation with binarization
(c) Linear weighted sum
(d) Linear weighted sum with binarization
Realization for a Binary Image:
L NRBD S l ti ( t ) Larger-NRBD Selection (cont.)
z
We propose a one-pass algorithm to replace the two-pass algorithm linear p p g weighted sum followed by binarization.z
We may call the process as largerz
We may call the process as larger- NRBD selection algorithm.Realization for a Binary Image:
L NRBD S l ti ( t )
Larger-NRBD Selection (cont.)
Realization for a Binary Image:
L NRBD S l ti ( t ) Larger-NRBD Selection (cont.)
))), (
), ( max(
(arg )
(P1.5 I D P1 D P2
I = NRBD NRBD
Take a Break
Take a Break
Outline Outline
z
Introductionz
Weighting-Based Digital Zoomz
Weighting Based Digital Zoom AlgorithmsA B d R t ti d R l
z Area-Based Restoration and Resample
z
Experimental Resultspe e ta esu tsz
ConclusionIntroduction Introduction
z
We assume an intensity value of a pixel is the integration of the light energy in a g g gy Charge-Coupled Device (CCD) grid.z
Based on the CCD grid mode and thez
Based on the CCD grid mode and the hypothesized boundary concept we canf
restore the infinite-high-resolution or continuous signal locally.g y
Introduction (cont.)
Introduction (cont.)
CCD Grid Mode and the Hypothesized
B d L li ti
Boundary Localization
z
We want to scale up the image two-by- two. We may divide a pixel in a low-y presolution image into four pixels to get a high-resolution image
high resolution image.
z
The pixel center in the low-resolutionimage locates on one pixel center in the high-resolution image.g g
CCD Grid Mode and the Hypothesized
B d L li ti
Boundary Localization
CCD Grid Mode and the
H th i d B d L li ti
Hypothesized Boundary Localization
(a) Area-based algorithm.
(b) Linear weighted sum
( ) g
algorithm.
Local Restoration Local Restoration
(d), (e), (f) : mirrors of (a), (b), (c)
Local Restoration (cont.) Local Restoration (cont.)
z
Use (a) for an example. To count the area of A1
2 2
2 1
1 1
1
) (
) 1
( )
(
R
R R
I P
I
I A
I A
P I
=
× +
×
−
= 1
IR1: Original intensity of P1 IR2: Original intensity of P2
2 2
)
(
Rz
Let the area of each pixel be 1R2 g y 2
) 1
( )
( 1 1 2
1 A
I A P
IR I R
−
×
= −
et t e a ea o eac p e be
) (
) 1
(
2 2
1
P I I
A
R =
Local Resampling for Scaling up by T
Two
z
Using the high-resolution area divided by the hypothesized boundary to calculate yp y the intensityz
Each high resolution pixel’s areaz
Each high-resolution pixel s area becomes 0.5z
Take (a) for exampleLocal Resampling for Scaling up by T ( t )
Two (cont.)
' '
'
) ( 0 5 )
( P A I A I
I
2 '
5 . 1
2 1
1 1
1
) (
) 5
. 0 ( )
(
R
R R
I P
I
I A
I A
P I
=
× +
×
−
=
2 '
2
)
( P I
RI =
Area-based algorithmg
1x 2x
4x
Outline Outline
z
Introductionz
Weighting-Based Digital Zoomz
Weighting Based Digital Zoom AlgorithmsA B d R t ti d R l
z
Area-Based Restoration and Resamplez Experimental Results pe e ta esu ts
z
ConclusionExperimental Results Experimental Results
z Visualization results
z Digital zoom on red g rectangle
z Five methods will be Five methods will be
compared.
Experimental Results (cont.) Experimental Results (cont.)
(a) Nearest-neighbor pixel copy (b) Bili i t l ti
(b) Bilinear interpolation (c) Linear weighted sum (d) Sigmoid weighted sum (d) Sigmoid weighted sum (e) Area-based algorithm
Experimental Results (cont.) Experimental Results (cont.)
(a) Original image
( ) g g
(b) Nearest-neighbor pixel copy
(c) Bilinear interpolation (d) Larger-NRBD
selection
Experimental Results (cont.) Experimental Results (cont.)
(a) Original image
( ) g g
(b) Nearest-neighbor pixel copy
(c) Bilinear interpolation (d) Larger-NRBD
selection
Experimental Results (cont.) Experimental Results (cont.)
(a) Original image
( ) g g
(b) Nearest-neighbor pixel copy
(c) Bilinear interpolation (d) Larger-NRBD
selection
SNR and Sharpness Comparison SNR and Sharpness Comparison
Original Image 4x4 nearest neighbor pixel copy Original Image 4x4 nearest neighbor pixel copy
Sharpness: 153 037 221 SNR: 9 551811 Sharpness: 153,037,221 SNR: 9.551811
Sharpness: 99,597,786 SNR: Signal-to-Noise Ratio
SNR and Sharpness Comparison ( t )
(cont.)
4x4 bilinear interpolation 4x4 linear weighted sum 4x4 bilinear interpolation 4x4 linear weighted sum
SNR: 12 807445 SNR: 12 730128
SNR: 12.807445
Sharpness: 23,033,292
SNR: 12.730128
Sharpness: 26,264,322
SNR and Sharpness Comparison ( t )
(cont.)
4x4 sigmoid weighted sum 4x4 area-based algorithm 4x4 sigmoid weighted sum 4x4 area based algorithm
SNR: 12 655672 SNR: 13 046390
SNR: 12.655672
Sharpness: 23,194,836
SNR: 13.046390
Sharpness: 75,389,265
Outline Outline
z
Introductionz
Weighting-Based Digital Zoomz
Weighting Based Digital Zoom AlgorithmsA B d R t ti d R l
z
Area-Based Restoration and Resamplez
Experimental Resultspe e ta esu tsz Conclusion
Conclusion Conclusion
z
Nearest-neighbor pixel copyz
Advantage: fast, simple g , p
z
Disadvantage: blocky, useless in scaling up process
process
z
Bilinear interpolationz
Advantage: fast, simple
z
Disadvantage: blocky and blurry effects on g y y
edges
Conclusion (cont.) Conclusion (cont.)
z Linear weighted sum
z
Advantage: efficient, fast, no blocky effects on edges, good visualization
z
Disadvantage: not sharp enough on edges
z Sigmoid weighted sum
z
Advantage: a user-defined parameter for tuning, sharpness larger than bilinear interpolation even linear weighted sum
z
Disadvantage: SNR smaller than bilinear
interpolation
Conclusion (cont.) Conclusion (cont.)
z
Larger-NRBD selectionz
Advantage: suitable for binary images g y g
z
Disadvantage: only for binary images
A b d l ith
z
Area-based algorithmz
Advantage: good SNR and sharpness values
z