Environment Setting

(1)

VFX Project #3: MatchMove

Digital Visual Effects, Spring 2016

2016/5/4

(2)

Overview

Video

CGI Animation

Composite Video

Input

Output

Matchmove is a technique of estimating camera parameters from an input video sequence so that computer generated imagery (CGI) could be seamlessly inserted into the sequence.

Example

2

(3)

Environment Setting

• Blender

– Version 2.6.2 http://download.blender.org/release/Blender2.62/

– Using version higher than 2.6.6 might get error while running voodoo python scripts

• Voodoo

– Version 1.2.0 http://www.viscoda.com/en/voodoo-download

• Other options

– Free: Blender + ICARUS

– Pay: boujou + 3D Max, boujou + Maya, …

• 3D models

– *.obj, *.3ds…

(4)

Flowchart

Video

Blender

Image sequences

Calibration

Camera parameters

3D models

Edit Pose/Position

Animation Output Video

Composition

Voodoo

Blender

Load virtual camera Split

(5)

• Three main stages:

1. In Blender: Split video into image sequences

• Input: video

• Output: image sequence (*.tga)

2. In Voodoo: Camera calibration/tracking

• Input: image sequence (*.tga)

• Output: voodoo python script (*.py)

3. In Blender: Combine video and 3D model

• Input: 3D model, video, voodoo python script (*.py)

• Output: video with 3D model

4. (Option) Edit your video with other software (ex. Adobe After Effects, VideoStudio, PowerDirector, Final Cut)

Flowchart (detailed)

(6)

Stage 1: Get Image Sequence

Video

Blender

Image sequences

Calibration

3D models

Composition Load virtual camera

Split

(7)

Stage 1: Get Image Sequence

1. Open Blender 2. Load video file:

– Change window type to “Video Sequence Editor”

– Select Add →Movie – Put the strip in Layer 1

3. Do sequence:

– Frame

• Set number of frames and resolution of frames

• Choose output file type (Targa, *.tga)

– Time interval: select start and end of the sequence – Choose output location

4. Click “Animation” button

(8)

Stage 1: Get Image Sequence

Start up Blender

(9)

Stage 1: Get Image Sequence

2. Change to Video Sequence Editor

1. Click to change window type

Change window type

(10)

Stage 1: Get Image Sequence

Add video

Select

Add  Movie

(11)

Stage 1: Get Image Sequence

Select input video

~

1. Select file directory

2. Select input file 3. Add

(12)

Stage 1: Get Image Sequence

Edit layers (Right click and drag, left click to set)

1. Delete unnecessary layer (audio)

2. Right-click to drag the strip to the “1^st Frame” in Layer 1

(13)

Stage 1: Get Image Sequence

Do video setting

1. Set

a. Resolution

b. Start/End Frame

2. Set file types to Targa(TGA)

y

3. Modify output directory

(14)

Stage 1: Get Image Sequence

Render image sequence

Click Animation

You’ll see playing video and

increasing frame number

p y

Finish!

You can find the image sequences in output directory

(15)

Stage 2: Camera Calibration

Video Image sequences

Calibration

3D models

Composition

Voodoo

(16)

1. Open Voodoo

2. Open image sequence:

– Select File  Open  Sequence – Select the first frame

– Set Move Type to “free move”

3. Track:

– Click Track button

4. Export Python script:

– Select File  Save  Blender Python Script – Save .py file (Blender 2.5x and higher)

– Export all

Stage 2: Camera Calibration

(17)

Stage 2: Camera Calibration

Use Voodoo to load image sequence

p q

Select

File  Open  Sequence

(18)

Stage 2: Camera Calibration

Choose and set sequence

1. Select the first frame (ex: 0001.tga)

2. Select “free move”

(general moving of camera)

(19)

Stage 2: Camera Calibration

Tracking features

Click “Track” button

(20)

Stage 2: Camera Calibration

Save results to “Blender Python Script”

Select

File  Save  Blender Python Script

g Select

“Blender 2.5x and higher”

Select “Export all”

(21)

Stage 3: Combine Video and 3D Model

Calibration

3D models

Composition

Blender

(22)

• Stage 3-1: Load virtual camera (python script from voodoo)

• Stage 3-2: Show background images

• Stage 3-3: Load 3D model

• Stage 3-4: Set model animation

• Stage 3-5: Render video

Stage 3: Combine Video and 3D Model

(23)

Stage 3: Combine Video and 3D Model

Calibration

3D models

Composition

Blender

(24)

Stage 3-1 ~ 3-2

1. Open Blender

2. Delete default objects 3. Load Python Script

– Change Window Type to “Text Editor”

– Select Text  Open Text Block

– Select the *.py file (exported from Voodoo) – Click “Run Script”

4. Load Background Images

– Change window type to “3D View”

– Select View  Cameras  Set Active Object as Active Camera – Load background images

– Set the parameters of background images

(25)

Stage 3-1: Load Virtual Camera

Delete default objects

Press “a” twice to select all objects, then press “Delete” on your keyboard

(26)

Stage 3-1: Load Virtual Camera

Load python script

Change window type to

“Text Editor”

Select “Text  Open Text Block”

Select the Python script exported by Voodoo

(27)

Stage 3-1: Load Virtual Camera

Run script

1. Press “Run Script”

2. Return to “3D View”

(28)

Stage 3-2: Show Background Images

Show Property Panel

p Turn on

“Properties” Panel

(29)

Stage 3-2: Show Background Images

g g

Tick-off

“Background Images”

Press “Open”

Select

the first frame

(30)

Stage 3-2: Show Background Images

Change

background type

Select

“Image Sequence”

Set

1. Number of frames 2. Start frame (1)

Enable “Auto Refresh”

(31)

Stage 3-2: Show Background Images

Set resolution

Set

1. Number of frames

2. Resolution

(32)

Stage 3-2: Show Background Images

Select

“voodoo_render_cam”

Select Voodoo camera

(33)

Stage 3-2: Show Background Images

Set view

Switch view to

“View Persp/Ortho”

Switch view to

“Front”

(34)

Stage 3-2: Show Background Images

Set active camera

Set

Cameras  Set Active Object as Camera

(35)

Stage 3-2: Show Background Images

Check background

You can drag here to see the camera motions

Make sure “voodoo_render_cam” is selected Make sure “voodoo_render_cam” is selected

(36)

Stage 3: Combine Video and 3D Model

Calibration

3D models

Composition

Blender

(37)

Stage 3-3 ~ 3-4

• Load models

– File  Import  *.obj

• Edit model poses/motions in the video

– Object  Animation  Insert keyframe (or press “I”) – Fine tuning

(38)

Stage 3-3: Load 3D Model

Load models

Select

File  Import Wavefront

(you can also use other file format)

(39)

Stage 3-3: Load 3D Model

Set poses / positions for models

p

You can set the transform of your objects to edit their positions and poses

(40)

Stage 3-3: Load 3D Model

Fine tuning

This step needs lots of fine tuning

( j p )

You can drag here to see whether your tuning is correct or not

(is the object located at the correct position?)

(41)

Stage 3-4: Set Model Animation

Set keyframes

1. Select the frame that you want to insert a key frame

2. Select

Object  Animation  Insert Keyframe..

3. Repeat multiple times

Frame 1

Frame 80

Frame 150

(42)

Stage 3-4: Set Model Animation

Fine tuning

y

You can fine-tune the

transformation of objects in a keyframe here

(43)

Stage 3: Combine Video and 3D Model

Calibration

3D models

Composition

Blender

(44)

Stage 3-5

• Add image sequence

– Change window type to “Video Sequence Editor”

– Select Add  Images and select all images – Drag the strip to the “1^st Frame” in Layer 1

• Add scene

– Select Add  Scene

– Drag the scene strip to the “1^st Frame” in Layer 2 – Change scene property to “Alpha Over”

– Set parameters of output video

• Click “Animation”

(45)

Stage 3-5: Render Video

Set to Video Sequence Editor Add image sequence (Video Channel)

Select

Add  Image

(46)

Stage 3-5: Render Video

This time, select all frames (Press A)

(47)

Stage 3-5: Render Video

Edit Video Layer (Right click and drag, left click to set)

Drag the strip to the

“1^st Frame” in Layer 1

(48)

Stage 3-5: Render Video

Add scene layer

Select

Add  Scene  Scene

Edit scene layer (Right click and drag, left click to set)

Drag the strip to the

“1^st Frame” in Layer 2

(49)

Stage 3-5: Render Video

Show Properties Panel

Select

View  Properties

Select

Blend  Alpha Over You will

see the Properties

Panel

Set Blend Type

(50)

Stage 3-5: Render Video

Select Premultiply

Enable!

Set scene camera

Set scene camera to

“voodoo_render_cam”

(51)

Stage 3-5: Render Video

Check output setting again!

(52)

Stage 3-5: Render Video

Preview your video

Change the viewing type

(53)

Stage 3-5: Render Video

Select output format

File path

Press ! Finish !!

Oh yeah!!

Final Output

Choose H.264

(54)

Demo

• https://www.youtube.com/watch?v=DzTiVhdxJk0

• https://www.youtube.com/watch?v=QU4pPnsqbeM

• https://www.youtube.com/watch?v=1f3fMV3v7Xg

• https://www.youtube.com/watch?v=B8LN9nMM3Wg

• https://www.youtube.com/watch?v=BM5EmTMjFy0

• More examples:

– http://ntuvfx.csie.org/vfx/2016/

– http://192.168.1.110/vfx/2016/

(55)

• You need to submit:

1. Artifact video: final video with CGI 2. Raw video: video without CGI

3. Report in html/pdf format

• Upload a compressed file (including raw video and report) and your artifact to the submission website.

– http://ntuvfx.csie.org/vfx/2016/proj3_submit.php – http://192.168.1.110/vfx/2016/proj3_submit.php

Submission

(56)

• Camera motion

• Model motion: ex. moving on the ground vs. flying

• Video completeness: ex. story

• Creativity

• Bonus: voting

Grading Criterion

(57)

FAQ

• Voodoo會當掉！

– 拍攝的影片內容差異太大

– 調整tracking的方式

• 為什麼我在Blender中的Background和Feature沒辦法配合?

– 記得要先選取 voodoo_render_cam

– 設定Cameras  Set Active Object as Camera

• 3D models的位置/動作好難調整…

– 多利用座標系相對位置以及不同視角會有幫助, 不過還是需要一些時間

– 設定動作時, 盡量避免大角度或大範圍的interpolation, 多設一些 key frames

• 為什麼最後做出來的動畫Model是黑的?

– 記得在場景裡加盞燈

• 我該去哪裡找3D models?

– 網路上免費資源非常多!

– 例如: http://www.3dm3.com/modelsbank/

http://www.sharecg.com/

(58)

FAQ

調整設定View  controls

(59)

FAQ

• Use different views to adjust the positions and poses of objects

Camera view

Front view

Top view Set

view

(60)

FAQ

• Add lights

記得開燈!

Add  Lamp

(61)

Document

• Blender official website

– http://www.blender.org/education-help/

• Voodoo document website

– http://www.viscoda.com/index.php/en/voodoo-manual