Noisy Video Super-Resolution

Noisy Video Super-Resoluti on

Feng Liu, JinjunWang,ShenghuoZhu (MM’08)

University of Wisconsin-Madison, NEC Laboratories America, Inc.

第一組： 資訊四 B95902105 黃彥達 資訊碩一 R98922046 蔡旻光 網媒碩二 R97944012 鄒志鴻

Outline

 Introduction

 Goal

 File Format

 Noise Reduced Image

 Proposed Approach

 Motion Estimation & Estimated Super-Resol ution Result

 Implementation

 Result

 Conclusion

2

Introduction

 Low-quality videos often not only have lim ited resolution but also suffer from noise

 In fact, the requirements of de-noising & super- resolution is quite similar

 This paper present a unified framework wh ich achieves simultaneous video de-noisin g and super-resolution algorithm by some m easurements of visual quality

3

Goal



Refine low-quality videos from YouTub e, and make the video better effects, which has better quality by human eye s.



Input is low-quality and noise-includ

ed (block effects or somewhat noise)

videos

Noise-Reduced Image

mv-SAD Gaussian- space

Gaussian- time

| p(I,j) – p(i’, j’) | >

threshold

Gaussian Space

Frame t

Pixel(I ,j)

Standard deviation Set Mean

= 0

Motion Vector

Frame t Pixel ( i , j , t)

Frame Pixel ( i + mv_i , j + mv_j t+1 , t+1)

(mv_i , mv_j)

Gaussian Time

Frame t - 2

Frame t - 1

Frame t

Space Gaussian

Time Gaussian

Pixel(I ,j)

Frame t+1

Frame t+2

Frame t

Noise-Reduced Image

Before After

Proposed Approach – 1 / 4

 Consider the visual quality with respect t o the following 3 aspects:

 Fidelity Preserving

▪ To achieve similar high-resolution result

 Detail Preserving

▪ Enhanced details (edge)

 Spatial-Temporal Smoothness

▪ Remove undesirable high-frequency contents (e.g. jitter)

10

Proposed Approach – 2 / 4

 Fidelity Preserving

 Conventional metrics:

▪ Measure fidelity by the difference between I_h & I_l would be problematic & waste useful time-space information in video

 Proposed metrics:

▪ Estimate an approximation of super-resolution results from space-time neighboring pixels

▪ The fidelity measurement:

see next page for details noised

11

Proposed Approach – 3 / 4

 Detail Preserving

 Enhanced details (edge)

 Contrast preserving

▪ Human visual system is more sensitive to contrast than pixel values

▪ Gradient fields of I^h & should be close

,where W_k is one or zero if the patch_k with/o edges (canny detector)

12

Proposed Approach – 4 / 4

 (Spatial-Temporal) Smoothness

 Smooth results are often favored by the human system

 Encourage to minimize:

 A 2-D Laplace filter may be

Spatial-temporal Laplacian

OR

13

An Optimization Problem

 Proposed Measurements

 A quadratic minimization problem to solve (AX = b):

Contrast Similarity

Detail Information(edge)

Spatial-Temporal Smoothness

14

Implementation – 1 / 2

Input_low

Ih^~

Ih^~

Gh^~

Ih^~

) 1 , ( )

1 ,

(I^~ t w I^~ t

w_mv ^h _mv ^h

X =

I

6 -1 … -1 -1 6 -1 … -1

-1 6 -1 … -1^Laplacian

Gradient

-1 0 1 … 1 -1 0 1 … 1 -1 0 1 … 1

Minimize Edge

Motion Estimation Motion Estimation

++

Result (X)

Fidelity

Bilateral filter

wmv

fidv

fidg

dt

sm

15

Implementation – 2 / 2

 Adjustments for the weight terms

 The measurement term is more emphasized if the w eight is larger

 By iteratively experiments for our test data, we took

 However, we found that for different videos, the best weight sets may be also different

3 . 0 ,

1 . 0 ,

1 ,

1   

 _{fidg dt sm}

fidv   



16

Result



352 x 288 Result

17

Conclusion

 The proposed framework formulates noisy vi deo super-resolution as an optimization pr oblem, aiming to maximize the visual quali ty of the result

 The measurements of fidelity-preserving, d etail-preserving and smoothness are consid ered to maximize the visual quality result s

18

Thank you!!

19