Interactive Foreground Extraction using Iterated Graph Cuts

GrabCut

Interactive Foreground Extraction using Iterated Graph Cuts

Carsten Rother

Vladimir Kolmogorov Andrew Blake

GrabCut GrabCut

Interactive Foreground Extraction Interactive Foreground Extraction

using Iterated Graph Cuts using Iterated Graph Cuts

Carsten Rother Carsten Rother

Vladimir Kolmogorov Vladimir Kolmogorov

Andrew Blake Andrew Blake

Microsoft Research Cambridge - UK

Photomontage

Photomontage

Photomontage

Problem Problem Problem

Fast &

Accurate ?

What GrabCut does What GrabCut does What GrabCut does

User Input

Result

Magic Wand

(198?)

Intelligent Scissors

Mortensen and Barrett (1995)

GrabCut

Regions Boundary Regions & Boundary

Framework Framework Framework

Input: Image

Output: Segmentation

Parameters: Colour ,Coherence Energy:

Optimization:

Graph Cuts

Boykov and Jolly (2001)

Graph Cuts Graph Cuts

Boykov

Boykov and Jolly (2001) and Jolly (2001)

Image Image

Min Cut Min Cut

Cut: separating source and sink; Energy: collection of edges Min Cut: Global minimal enegry in polynomial time

Foreground Foreground

(source) (source)

Background Background

(sink)

(sink)

Iterated Graph Cut Iterated Graph Cut Iterated Graph Cut

User Initialisation

K-means for learning colour distributions

Graph cuts to infer the segmentation

?

1 2 3 4

Iterated Graph Cuts Iterated Graph Cuts Iterated Graph Cuts

Energy after each Iteration Result

Gu ara

nte ed to co nve

rge

Colour Model Colour Model Colour Model

Gaussian Mixture Model (typically 5-8 components)

Foreground &

Background

Background

Foreground

Background

G R

G Iterated R

graph cut

Coherence Model Coherence

Coherence Model Model

An object is a coherent set of pixels:

Blake et al. (2004): Learn jointly

Moderately straightforward examples

Moderately straightforward Moderately straightforward

examples examples

… GrabCut completes automatically

Difficult Examples Difficult Examples Difficult Examples

Camouflage &

Low Contrast Fine structure No telepathy

Initial Rectangle

Initial

Result

Evaluation – Labelled Database Evaluation Evaluation – – Labelled Database Labelled Database

Available online: http://research.microsoft.com/vision/cambridge/segmentation/

Comparison Comparison Comparison

GrabCut Boykov and Jolly (2001)

User Input

Result

Summary Summary Summary

Magic Wand (198?)

Intelligent Scissors

Mortensen and GrabCut

Rother et al.

Graph Cuts Boykov and

LazySnapping

Li et al. (2004)