Live Migration of Virtual Machines
Christopher Clark, Keir Fraser, Steven Hand, Jacob Gorm Hanseny,
Eric July, Christian Limpach, Ian Pratt, Andrew Wareld
Index
• Motivation
• Migration
• Pre-copy migration
• Challenge in Pre-copy
• Implementation
• Managed Migration(Xen)
• Self Migration
• Experiment
• Optimization
Motivation
• Why we need live migration?
We want to move VM without interrupting V M service.
Products:
• Xen 2.0 : live migration
• VMware:VMotion
Migration
• Two important consideration:
1. Downtime
2. Total migration time
• All migrations mechanism can be divided into three phases.
1. Push phase
2. Stop-and-copy phase 3. Pull phase
Migration
Example:
• Stop-and-copy
=> Only stop-and-copy phase
• Pure demand-migration
=> Stop-and-copy phase and Pull phase
• Pre-copy
=> Push phase and Stop-and-copy phase
Pre-copy migration
Challenge in Pre-copy
• Every VM have some set of pages which updat es very frequently.
• And the Sets size would influence downtime o f the migration.
ÞWritable Working Sets
• The bandwidth of network also impact the mi gration time.
Implementation
• There are two kinds of implementation of migration.
– Managed Migration
• The physical machine has a management VM. (Xen)
– Self Migration
• There are three conditions to change into stop-and-c opy phase.
– The dirty rate is bigger than the upper bound.
– The size if working set is small enough.
– The limit of the rounds.
pages dirty
Current
#
pages dirty
Privious rate #
Dirty
Managed Migration(Xen)
1. 1st round:
– Copy all memory pages to destination machine.
– Replace original page table with shadow page table whose all pages are marked read-only .
– Create a dirty bit map for the VM.
2. 2nd-(n-1)th round:
– During the pages transferring, if VM want to modify a page, it will invoke Xen to set the appropriate bit in the dirty bit map.
– Dirty pages will be resend again.
– Reset dirty bit map for the next round.
3. nth round:
– When the Dirty rate is bigger than up bound, begin to do stop-and-copy.
Self Migration
1. 1st round:
– Copy all memory pages to destination machine.
– Mark all virtual address as write-protect.
– The OS create a dirty bit map(used to tracking physical pages) to memory the modified pag es.
2. 2nd-(n-1)th round:
– During the pages transferring, if write faults occur, OS will set the appropriate bit in the dirt y map.
– Dirty pages will resend again.
– Reset dirty bit map for the next round.
3. nth round:
– When the dirty rate is bigger than the upper bound, OS would disable all activities.
4. Copy all dirty pages to shadow buffer.
5. Transfer the buffer and ignore new page updates.
Web Server migration
SPECweb99 benchmark
A worst case example
Optimization
• Dynamic Rate-Limiting
• Rapid Page Dirtying
• Para-virtualized Optimization 1. Ballooning mechanism
2. Stunning rogue Process
