Course overview
Digital Visual Effectsg Yung-Yu Chuang
Logistics
• Meeting time: 2:20pm-5:20pm, Wednesday
Cl CSIE R 104
• Classroom: CSIE Room 104
• Instructor: Yung-Yu Chuang (cyy@csie.ntu.edu.tw)
• Teaching assistants: TBA
• Textbook: Readings from books journals and
• Textbook: Readings from books, journals and proceedings. Richard Szeliski’s Computer Vision:
Algorithms and Applications Algorithms and Applications.
• .Webpage: (user name/password) http://www csie ntu edu tw/ cyy/vfx http://www.csie.ntu.edu.tw/~cyy/vfx
• Mailing list: vfx@cmlab.csie.ntu.edu.tw subscribe via https://cmlmail.csie.ntu.edu.tw/mailman/listinfo/vfx/
This course is NOT about … This course is NOT about …
It isn’t about photography
It isn’t about 3D animations It isn’t about watching movies
It isn’t about physical effects It’s not about industrial tricks
You will learn more about Taylor and Poisson than Lucas and Spielberg in this course If you hear Lucas in the class and Spielberg in this course. If you hear Lucas in the class, it is more likely to refer to Bruce Lucas, not George Lucas.
computer graphics computer
vision
graphics
image processing processing
Prerequisites
• It is a must that you have programming experiences
experiences.
• It is a must that you have basic knowledge on li l b d b bilit
linear algebra and probability.
• It is a plus if you have background knowledge on computer vision, image processing and computer graphics.
• It is a plus if you have access to digital cameras and camcorders.
The vfx course
h t th ht
h t th f what you thought
you will do what other students
think you do what other professors
think you do
what you actually do
The vfx course Warning from previous students
‧請學期初老師要多提醒這門課的困難 度請興趣或實力不足的同學勿修，否 則就會像我一樣停修 XD
則就會像我一樣停修 XD
This course is about … This course is about …
Digital Visual Effects
Reality? Retouching
Iraq War, LA Times, April 2003
Digital photomontage
Bush campaign’s TV AD, 2004
Texture synthesis and inpainting Domestic example
The Liberty Times 2007 12 17
2007.12.17
Special effects Special effects
Stop action
The execution of Mary 1895 The execution of Mary, 1895
Glass shot Rear projection
stop motion
King Kong 1933 King Kong, 1933
Special effects (make-up) Special effects (physical effects)
Special effects (miniature) Special effects (matte painting)
Lord of the Rings Illusion - forced perspective
Computer-generated model Visual effects 100 Years
Production pipeline Production pipeline
Production pipeline
Preproduction Production
Effects
Artwork Reference Stills Plates
Storyboard
Post-production
Editing
VFX g
Music Music
Sound
Film
Preproduction
Storyboard Storyboard
Preproduction
Artwork
Preproduction
Reference & Research Reference & Research
Production
Shooting Shooting
Post-production Visual effects production
Preproduction Production Effects
Artwork Reference Stills Plates
Scanning Storyboard
Post-production
Matchmove and Matchmodel Model Geometry
Effects Animation
Character Animation
Ski i
Lighting and Rendering Skeleton Textures
M i l
Skinning Rendering
Compositing
Assets Shots
Materials
Assets Shots
Film Recording
Visual effects post-production
A case study A case study
405: The Movie
• This movie were created solely by two visual effects artists in the year of 2000 It was a process effects artists in the year of 2000. It was a process that took over three months of nights, weekends and any spare moments that they could find.
and any spare moments that they could find.
• http://www.405themovie.com/
405: The Movie
Making of 405
Step 0: script and shooting plan
Making of 405
Step 1: shooting
two days with a Canon Optura DV camera with progressive mode.
a 70-minute raw footage initial editing
pickup shots
pickup shots
Making of 405
Step 2: building CG world
total 62 shots, 42 enhanced with digital VFX.
19 shots are entirely digital creations.
plane, two cars, freeway background are digital
photo-based 3D environment
Making of 405
Real cars were used for close-up and interior shots
A low-resolution mesh scanned b b
by a cyberscanner.
Mapped with photographs.
Making of 405
DC-10 plane took a month to model in details for the to model in details for the needs of close-up shots.
59 objects, 142,439 polygons
reference modeling material painting
Making of 405
Step 3: traffic clearing clean plate
close-up shots
inpainting inpainting
Making of 405
Step 4: compositing
h t ith th hi l t di till i b k d shot with the vehicle standing still in a backyard
Making of 405
Making of 405
Step 5: fine touchup
3D hat compositing and inpaintingp g p g
Making of 405
Step 6: music and delivery
Bloody Omaha
Topics we plan to cover p p
Camera
Canon 10D Canon 10D
High dynamic range imaging/display
Image warping/morphing
someone very famous
someone not
th t f famous
that famous
video
Image warping/morphing
Tracking
Feature tracking Feature tracking
Image stitching
MatchMove
Move matching using scene planes Move matching using scene planes
Matchmove
Move matching using scene planes Move matching using scene planes
Matchmove
Move matching using scene planes Move matching using scene planes
Photo tourism
Video matching
MOCO (Motion control camera) Matrix MOCO (Motion control camera) Matrix
Video matching
Video matching Video matching
Stop motion Stop motion
Matting and compositing
Titanic
Matting
Object selection
LazySnapping LazySnapping
Image-based modeling
photogrammetric modeling and projective texture-mapping photogrammetric modeling and projective texture-mapping
Image-based modeling
photogrammetric modeling and projective texture-mapping photogrammetric modeling and projective texture-mapping
Image-based modeling
photogrammetric modeling and projective texture-mapping photogrammetric modeling and projective texture-mapping
Image-based modeling
Tour into a picture Tour into a picture
Image-based modeling
Tour into a picture Tour into a picture
3D photography (active)
Cyberware whole body scanner Cyberware whole body scanner
3D photography (active)
Photometric stereo Photometric stereo
3D photography (passive)
left right
Stereo
depth Stereo
Image-based rendering
Surface lightfield Surface lightfield
View interpolation
Bullet time video Bullet time video
View interpolation
High Quality Video View Interpolation High-Quality Video View Interpolation
Image manipulation
GraphCut Texture GraphCut Texture
Image manipulation
Poisson blending Poisson blending
Stereoscopic films Making face
Spacetime face Gollum
Virtual human Video rewrite
Trainable videorealistic speech animation Trainable videorealistic speech animation
Inpainting (wire removal)
Inpainting Inpainting
Texture synthesis/replacement
Texture replacement Texture replacement
Semi-automatic matte painting
Image analogies Image analogies
Video editing
Flow based video editing Flow-based video editing
Grading (subject to change)
• 3 programming assignments (60%)
HDR I i (18%) – HDR Imaging (18%) – AutoStitch (24%)
M t hM (18%) – MatchMove (18%)
• Class participation (5%)
• Final project (35%)
– Research – System – Film
High dynamic range imaging
From past semesters (鄭逸廷 陳柏叡) From past semesters (吳侑親,張書瑋)
From past semesters (王瑋馥,余雁雲) AutoStitch
AutoStitch
羅聖傑 羅聖傑
連奕婷 張宇蓓 連奕婷 張宇蓓
MathMove
梁家愷 鐘志遠 姜任遠 林立峯
梁家愷 鐘志遠 姜任遠 林立峯
楊宗碩 林柏劭 翁憲政 洪韶憶
Final projects from a similar course in Georgia Tech.
Life in Paints
Life in Paints, GaTech DVFX 2003
Tour into pictures
Making of Life in Paints
In Your Face
In Your Face, GaTech DVFX 2002
Stop action
Making of In Your Face
Final projects from the past.
Final projects from the past.
YoYo Flight Making of YoYo Flight
That’s it for today!
• Don’t forget to subscribe the mailing list.
Ch k h b i
• Check out the course website.
