Course overview
Digital Visual Effects Yung-Yu Chuang
Logistics
• Meeting time: 2:20pm-5:20pm, Wednesday
• Classroom: CSIE Room 104
• Instructor: Yung-Yu Chuang (cyy@csie.ntu.edu.tw)
• Teaching assistants:
• Textbook: Readings from books, journals and proceedings. Richard Szeliski’s Computer Vision:
Algorithms and Applications. Richard Radke’s Computer Vision for Visual Effects.
• Webpage: (user name/password: ***/***) http://www.csie.ntu.edu.tw/~cyy/vfx
This course is NOT about …
It isn’t about photography
It isn’t about 3D animations
It isn’t about watching movies
It isn’t about physical effects
It’s not about industrial tricks
computer vision
computer graphics
image processing
You will learn more about Taylor and Poisson than Lucas and Spielberg in this course. If you hear Lucas in the class, it is more likely to refer to Bruce Lucas, not George Lucas.
Prerequisites
• It is a must that you have programming experiences.
• It is a must that you have basic knowledge on linear algebra and probability.
• It is a plus if you have background knowledge on computer vision, image processing and
computer graphics.
• It is a plus if you have access to digital cameras and camcorders.
The vfx course
what you actually do
what you thought you will do
what other students think you do
what other professors think you do
Be cautious!
Warning from previous students
‧請學期初老師要多提醒這門課的困難
度請興趣或實力不足的同學勿修，否
則就會像我一樣停修 XD
This course is about …
Digital Visual Effects
Digital Visual Effects
Deadpool
Deadpool
Life of Pi
Life of Pi
獨自一人拍和十三人的戲
要把身材高大的甘道夫和小矮人們拍攝在一起，我們是沒法在同一個片場的。和 我一起拍攝的只有柱子上貼著的13張他們的照片，後面還有一個小燈，哪個角 色說話了燈就亮起來。想像一下你在拍一場和13個人一起演的戲，但你卻只有 獨自一人。這真的會把你的演技推到極限。我哭了，真的，我當時真的哭了。然 後我還說出了聲：我認真演了一輩子不是為了跟這些照片對戲啊！
VFX of the Hobbit
Reality?
Retouching
Retouching
Retouching
Bush campaign’s TV AD, 2004
Texture synthesis and inpainting
Iraq War, LA Times, April 2003
Digital photomontage
Domestic example
The Liberty Times 2007.12.17
Special effects
Stop action
The execution of Mary, 1895
Glass shot
Rear projection
King Kong, 1933
stop motion
Special effects (make-up)
Special effects (physical effects)
Special effects (miniature)
Special effects (matte painting)
Lord of the Rings
Illusion - forced perspective
Computer-generated model
The Avengers
The Avengers (1978)
The Avengers (2012)
Visual effects 100 Years
Production pipeline
Production pipeline
Preproduction Production
Post-production
Artwork Reference Stills Plates
Storyboard
Editing VFX
Music
Sound
Film
Effects
Preproduction
Storyboard
Preproduction
Artwork
Preproduction
Reference & Research
Production
Shooting
Post-production
Visual effects production
Preproduction Production
Post-production
Artwork Reference Stills Plates
Model Geometry
Skinning
Effects Animation
Character Animation
Lighting and Rendering Compositing
Matchmove and Matchmodel
Assets Shots
Skeleton
Scanning
Film Recording Textures
Materials Storyboard
Effects
Visual effects post-production
A case study
405: The Movie
• This movie were created solely by two visual
effects artists in the year of 2000. It was a process that took over three months of nights, weekends and any spare moments that they could find.
• https://en.wikipedia.org/wiki/405_(film)
• An early example of digital filmmaking and the use of Internet as media
• Budget: $300 ($140 for tickets. The officer is acknowledged)
405: The Movie
Making of 405
Step 0: script and shooting plan
Making of 405
Step 1: shooting
two days with a Canon Optura DV camera with progressive mode.
a 70-minute raw footage initial editing
pickup shots
Making of 405
Step 2: building CG world
total 62 shots, 42 enhanced with digital VFX.
19 shots are entirely digital creations.
plane, two cars, freeway background are digital
photo-based 3D environment
Making of 405
Real cars were used for close-up and interior shots
A low-resolution mesh scanned by a cyberscanner.
Mapped with photographs.
Making of 405
DC-10 plane took a month to model in details for the needs of close-up shots.
59 objects, 142,439 polygons
reference modeling material painting
Making of 405
close-up shots
Step 3: traffic clearing
inpainting
clean plate
Making of 405
shot with the vehicle standing still in a backyard Step 4: compositing
Making of 405
Making of 405
Step 5: fine touchup
3D hat compositing and inpainting
Making of 405
Step 6: music and delivery
Bloody Omaha
Breakdown (Wolf of Wall Street)
Topics we plan to cover
Camera
Canon 10D
High dynamic range imaging/display
Image warping/morphing
someone very famous
someone not that famous
Image warping/morphing
someone very famous
someone not that famous
Image warping/morphing
Tracking
Feature tracking
Image stitching
MatchMove
Move matching using scene planes
Matchmove
Move matching using scene planes
Matchmove
Move matching using scene planes
Photo tourism
Video matching
MOCO (Motion control camera) Matrix
Video matching
Video matching
Matting and compositing
Titanic
Matting
Object selection
LazySnapping
Image-based modeling
photogrammetric modeling and projective texture-mapping
Image-based modeling
photogrammetric modeling and projective texture-mapping
Image-based modeling
photogrammetric modeling and projective texture-mapping
Image-based modeling
Tour into a picture
Image-based modeling
Tour into a picture
3D photography (active)
Cyberware whole body scanner
3D photography (active)
Photometric stereo
3D photography (passive)
Stereo
left right
depth
Image-based rendering
Surface lightfield
View interpolation
Bullet time video
View interpolation
High-Quality Video View Interpolation
Image manipulation
GraphCut Texture
Image manipulation
Poisson blending
Stereoscopic films
Making face
Spacetime face Gollum
Virtual human
Video rewrite
Trainable videorealistic speech animation
Inpainting (wire removal)
Inpainting
Texture synthesis/replacement
Texture replacement
Semi-automatic matte painting
Image analogies
Video editing
Flow-based video editing
Grading (subject to change)
• 3 programming assignments (60%)
– HDR Imaging (18%) – AutoStitch (24%) – MatchMove (18%)
• Class participation (5%)
• Final project (35%)
– Research – System – Film
High dynamic range imaging
From past semesters (鄭逸廷 陳柏叡)
From past semesters (吳侑親,張書瑋)
From past semesters (王瑋馥,余雁雲)
AutoStitch
AutoStitch
羅聖傑
連奕婷 張宇蓓
MathMove
梁家愷 鐘志遠
楊宗碩 林柏劭 翁憲政 洪韶憶
姜任遠 林立峯
