Subject: Xen + LVM + OS image - msg#00257

Hello all,

I have a setup that I have been testing to create a FC6 guest image on a
lvm logical volume (lv) and then use that to deploy new a new guest. I
wanted to get feedback about anything that I might have overlooked.

In a nutshell, each xen guest install has it's own lv for a root partition
and swap partition provided by the host. The guest installs are not using
lvm, they use the whole 'disk' as they see it, which is of course managed
by the host lvm.

To deploy a new FC6 guest, lvcreate new swap and data partitions and dd
the image lv to the new guest partition.

Then a copy of the /etc/xen/image is made, the mac address, uuid and name
are updated. Furthermore, in the new guest os the mac address and
networking settings must be updated.

An external USB2.0 drive that is managed by lvm is used for backups. Use
pvcreate and vgcreate to create a backup volume group (first time only)
and create logical volumes, shutdown the guests and dd the guest lv to
the backup volume group. Then use vgchange and vgexport to remove the
backup volume (pvscan and vgimport are used to re-attach the volume in the
future).

The following is a edited version of my more detailed notes.

I would really like to hear any suggestions or any major problems with
this,

Thanks!
Gordon Turner.



1. Host setup
-------------
- Install FC6 and xen on the host, using LVM and leaving space for the
guest logical volumes.
- Create logical volumes for xen hosts and create swap space:
lvcreate -L 1G -n ImageSwap00 VolGroup00
lvcreate -L 3G -n ImageData00 VolGroup00


2. Create image domain
----------------------
- Run installer:
[root@xen ~]# /usr/sbin/xenguest-install
What is the name of your virtual machine? image
How much RAM should be allocated (in megabytes)? 256
What would you like to use as the disk (path)? /dev/VolGroup00/ImageData00
Would you like to enable graphics support? (yes or no) no
What is the install location? http://192.168.1.199/fc6/

- During the guest install, remove default lvm partitioning and create
one filesystem that completely fills the available 'drive'. Ignore the
swap warning, it will be added later.

- After the first boot of the guest image, update fstab in the image to
include the swap space:
vi /etc/fstab
/dev/xvdb swap swap defaults 0 0

- Then halt the guest os:
halt

- In the host xen system, edit the /etc/xen/image file to shrink the
ram(optional) and add the swap:
vi /etc/xen/image
...
memory = "64"
disk = [ 'phy:/dev/VolGroup00/ImageData00,xvda,w', 'phy:/dev/VolGroup00/ImageSwap00,xvdb,w' ]

- Create the swap space:
mkswap /dev/VolGroup00/ImageSwap00

- Boot and test guest image, halt when done.


3. Create a new guest from image
--------------------------------
- Create lvm partitions:
lvcreate -L 1G -n ExampleSwap00 VolGroup00
lvcreate -L 3G -n ExampleData00 VolGroup00

- Map the ImageData00 partition:
kpartx -av /dev/VolGroup00/ImageData00

- Use dd to copy to the new partition:
dd if=/dev/mapper/ImageData00p1 of=/dev/VolGroup00/ExampleData00

- Create swap space:
mkswap /dev/VolGroup00/ExampleSwap00

- Unmap the ImageData00 partition:
kpartx -dv /dev/VolGroup00/ImageData00

- Mount the new data partition:
mount /dev/VolGroup00/ExampleData00 /mnt/Data

- In new host, edit the following files and change 'image' to new name:
vi /mnt/Data/etc/hosts
vi /mnt/Data/etc/sysconfig/network

- Also on new host, set the correct ip address and MAC address:
vi /mnt/Data/etc/sysconfig/network-scripts/ifcfg-eth0

- Unmount the new data partition:
umount /mnt/Data

- Copy the 'image' xen configuration file and edit it:
cp /etc/xen/image /etc/xen/example
vi /etc/xen/example


4. Create new Backup Volume Group
---------------------------------
- One time deal:
pvcreate /dev/sda
vgcreate VolBackup00 /dev/sda
vgchange -a n VolBackup00
vgexport VolBackup00


5. Backup Logical Volume
------------------------
- Plugin USB2.0 drive, wait for initialization(check dmesg) and import the
Volume Group:
pvscan
vgimport VolBackup00

- To create and backup to new logical volume:
lvcreate -L3G -n YYYYMMDD_NAME VolBackup00
dd if=/dev/VolGroup00/ImageData00 of=/dev/VolBackup00/YYYYMMDD_NAME

- To export the Volume Group and disconnect the drive:
vgchange -a n VolBackup00
vgexport VolBackup00


6. Notes and questions
----------------------
- My understanding is that using dd to copy lv around is safe as long as
the physical extents are the same, ie lvcreate should create volumes of
exactly the same size.

- Not sure the exact process of increasing a guest lv from the host, but
it should be possible.

- When a new copy of an image is copied, the ssh keys should be recreated
so that they are unique. Haven't looked at how to do that yet.