• 沒有找到結果。

Noisy Video Super-Resolution

N/A
N/A
Protected

Academic year: 2022

Share "Noisy Video Super-Resolution"

Copied!
23
0
0

加載中.... (立即查看全文)

全文

(1)

Noisy Video Super-Resoluti on

Feng Liu, JinjunWang,ShenghuoZhu (MM’08)

University of Wisconsin-Madison, NEC Laboratories America, Inc.

第一組: 資訊四 B95902105 黃彥達 資訊碩一 R98922046 蔡旻光 網媒碩二 R97944012 鄒志鴻

(2)

Outline

 Introduction

 Goal

 File Format

 Noise Reduced Image

 Proposed Approach

 Motion Estimation & Estimated Super-Resol ution Result

 Implementation

 Result

 Conclusion

2

(3)

Introduction

 Low-quality videos often not only have lim ited resolution but also suffer from noise

In fact, the requirements of de-noising & super- resolution is quite similar

 This paper present a unified framework wh ich achieves simultaneous video de-noisin g and super-resolution algorithm by some m easurements of visual quality

3

(4)

Goal

 Refine low-quality videos from YouTub e, and make the video better effects, which has better quality by human eye s.

 Input is low-quality and noise-includ

ed (block effects or somewhat noise)

videos

(5)

File Format

 .3gp file

 Frame rate: 15(our video) or 25

 Frame size: 176(w) * 144(h)

 MPEG-4 Part 12

 It is used on 3G mobile phones also can be played on

2G and 4G phones.

 Our video: 867 KB/ 98 sec

5

(6)

Noise-Reduced Image

mv-SAD Gaussian- space

Gaussian- time

| p(I,j) – p(i’, j’) | >

threshold

(7)

Gaussian Space

Frame t

Pixel(I ,j)

Standard deviation Set Mean

= 0

(8)

Motion Vector

Frame t Pixel ( i , j , t)

Frame Pixel ( i + mv_i , j + mv_j t+1 , t+1)

(mv_i , mv_j)

(9)

Gaussian Time

Frame t - 2

Frame t - 1

Frame t

Space Gaussian

Time Gaussian

Pixel(I ,j)

Frame t+1

Frame t+2

Frame t

Shot Detecti

on

(10)

Noise-Reduced Image

Before After

(11)

Proposed Approach – 1 / 4

 Consider the visual quality with respect t o the following 3 aspects:

Fidelity Preserving

▪ To achieve similar high-resolution result

Detail Preserving

▪ Enhanced details (edge)

Spatial-Temporal Smoothness

▪ Remove undesirable high-frequency contents (e.g. jitter)

11

(12)

Proposed Approach – 2 / 4

 Fidelity Preserving

Conventional metrics:

▪ Measure fidelity by the difference between Ih & Il would be problematic & waste useful time-space information in video

Proposed metrics:

▪ Estimate an approximation of super-resolution results from space-time neighboring pixels

▪ The fidelity measurement:

see next page for details noised

12

(13)

Proposed Approach – 3 / 4

 Detail Preserving

Enhanced details (edge)

Contrast preserving

▪ Human visual system is more sensitive to contrast than pixel values

▪ Gradient fields of Ih & should be close

,where Wk is one or zero if the patchk with/o edges (canny detector)

13

(14)

Proposed Approach – 4 / 4

 (Spatial-Temporal) Smoothness

Smooth results are often favored by the human system

Encourage to minimize:

A 2-D Laplace filter may be

Spatial-temporal Laplacian

OR

14

(15)

An Optimization Problem

 Proposed Measurements

 A quadratic minimization problem to solve ( AX = b):

Contrast Similarity

Detail Information(edge)

Spatial-Temporal Smoothness

15

(16)

Implementation – 1 / 3

Inputlow h

I~

Ih~

Gh~

Ih~

) 1 , ( )

1 ,

(I~ t w I~ t

wmv h mv h

X =

I

6 -1 … -1 -1 6 -1 … -1

-1 6 -1 … -1Laplacian

Gradient

-1 0 1 … 1 -1 0 1 … 1 -1 0 1 … 1

Minimize Edge

Motion Estimation Motion Estimation

++

Result (X)

Fidelity

Gaussian filter

wmv

fidv

fidg

dt

sm

16

(17)

Implementation – 2 / 3

....

,

1

500

1

11

x b s too slow

A

nn n

n

17

Ih~

Gh~

Ih~

) 1 , ( )

1 ,

(I~ t w I~ t

wmv h mv h

X =

I

6 -1 … -1 -1 6 -1 … -1

-1 6 -1 … -1Laplacian

Gradient

-1 0 1 … 1 -1 0 1 … 1 -1 0 1 … 1

Minimize Edge

Fidelity

fidv

fidg

dt

sm

!

!

! 2

1 A ' x A b s

x A

A t n n nt

by sparse least square solver

(18)

Implementation – 3 / 3

 Adjustments for the weight terms

The measurement term is more emphasized if the w eight is larger

By iteratively experiments for our test data, we took

However, we found that the best weight set may b e different for different videos

3 . 0 ,

1 . 0 ,

1 ,

1   

fidg dt sm

fidv

  

18

(19)

Result

 352 x 288 Result

19

(20)

Result

 352 x 288 Result

20

(21)

Result

 352 x 288 Result

21

(22)

Conclusion

The proposed framework formulates noisy video super-resolution as an optimization problem, aiming to maximize the visual quality

By exploiting the space-temporal information, we can estimate a better baseline than conven tional fidelity measurement

The properties include fidelity-preserving, d etail-preserving and smoothness are considere d to achieve the best visual quality results

22

(23)

Thank you!!

23

參考文獻

相關文件

• There are important problems for which there are no known efficient deterministic algorithms but for which very efficient randomized algorithms exist. – Extraction of square roots,

With the proposed model equations, accurate results can be obtained on a mapped grid using a standard method, such as the high-resolution wave- propagation algorithm for a

We would like to point out that unlike the pure potential case considered in [RW19], here, in order to guarantee the bulk decay of ˜u, we also need the boundary decay of ∇u due to

Then, we recast the signal recovery problem as a smoothing penalized least squares optimization problem, and apply the nonlinear conjugate gradient method to solve the smoothing

Then, we recast the signal recovery problem as a smoothing penalized least squares optimization problem, and apply the nonlinear conjugate gradient method to solve the smoothing

 High-speed sectioning images (up to 200 Hz) via temporal focusing-based widefield multiphoton microscopy.  To approach super-resolution microscopy

Rugged, high resolution, full-color, video-rate displays enable a multitude.

• If we know how to generate a solution, we can solve the corresponding decision problem. – If you can find a satisfying truth assignment efficiently, then sat is