GrabCut GrabCut

(1)

GrabCut GrabCut

Interactive Foreground Extraction Interactive Foreground Extraction

using Iterated Graph Cuts using Iterated Graph Cuts

Carsten Rother Carsten Rother

Vladimir Kolmogorov Vladimir Kolmogorov

Andrew Blake Andrew Blake

Microsoft Research Cambridge-UK Microsoft Research Cambridge-UK

GrabCut GrabCut

Interactive Foreground Extraction Interactive Foreground Extraction

using Iterated Graph Cuts using Iterated Graph Cuts

Carsten Rother Carsten Rother

Vladimir Kolmogorov Vladimir Kolmogorov

Andrew Blake Andrew Blake

Microsoft Research Cambridge-UK

(2)

Photomontage Photomontage Photomontage Photomontage

GrabCut – Interactive Foreground Extraction 1

(3)

video

(4)

Problem Problem Problem Problem

Fast &

Accurate ?

(5)

What GrabCut does What GrabCut does What GrabCut does What GrabCut does

User Inpu t

Result

Magic Wand

(198?)

Intelligent Scissors

Mortensen and Barrett (1995)

GrabCut

Regions Boundary Regions & Boundary

(6)

Framework Framework Framework Framework

Input: Image

Output: Segmentation

Parameters: Colour ,Coherence Energy:

Optimization:

(7)

Graph Cuts Graph Cuts

Boykov and Jolly (2001) Boykov and Jolly (2001)

Graph Cuts Graph Cuts

Boykov and Jolly (2001) Boykov and Jolly (2001)

Image Image

Min Cut Min Cut

Cut: separating source and sink; Energy: collection of edges Min Cut: Global minimal enegry in polynomial time

Foreground Foreground

(source) (source)

Background Background

(sink)

(8)

Iterated Graph Cut Iterated Graph Cut Iterated Graph Cut Iterated Graph Cut

User Initialisation

K-means for learning colour distributions

Graph cuts to infer the segmentation

?

(9)

1 2 3 4

Iterated Graph Cuts Iterated Graph Cuts Iterated Graph Cuts Iterated Graph Cuts

Energy after each Iteration Result

Gua ran

teed to

con verg e

(10)

Colour Model Colour Model Colour Model Colour Model

Gaussian Mixture Model (typically 5-8 components)

Foregrou nd &

Backgrou nd

Foregrou nd

Backgrou

G

nd

R

G

Iterated R

graph cut

(11)

Coherence Model Coherence Model Coherence Model Coherence Model

An object is a coherent set of pixels:

Blake et al. (2004): Learn jointly

(12)

Moderately straightforward Moderately straightforward examples

examples

Moderately straightforward Moderately straightforward examples

examples

… GrabCut completes automatically

(13)

Difficult Examples Difficult Examples Difficult Examples Difficult Examples

Camouflage &

Low Contrast Fine structure No telepathy

Initial Rectangl e

Initial Result

(14)

Evaluation – Labelled Database Evaluation – Labelled Database Evaluation – Labelled Database Evaluation – Labelled Database

Available online: http://research.microsoft.com/vision/cambridge/segmentation/

(15)

Comparison Comparison Comparison Comparison

GrabCut Boykov and Jolly

(2001)

Error Rate:

0.72%

Error Rate:

1.87%

Error Rate:

1.81%

Error Rate:

1.32%

Error Rate:

1.25%

Error Rate:

0.72%

User Input

Result

(16)

Summary Summary Summary Summary

Magic Wand (198?)

Intelligent Scissors

Mortensen and

Barrett (1995)

GrabCut Rother et al.

(2004) Graph

Cuts

Boykov and

Jolly (2001)

LazySnappi ng

Li et al.

(2004)