djdfy: Yet another libvirt/qemu HOWTO

Настроение:	content
Музыка:	Gamma Ray - Powerplant - Armageddon
Entry tags:	howto, libvirt, qemu

Yet another libvirt/qemu HOWTO

$Id: libvirt-howto.xhtml 435 2011-12-06 21:12:47Z djdfy $

STEP 0. Install debian

By default hard disk images stored by libvirt in /var/lib/libvirt/images, so you may want to create big separate partition for either /var/lib/libvirt or /srv, or /srv/libvirt. I prefer using /srv for purposes of "Drive D:" in Windows.

STEP 1. Install all requied software

Execute as root on the server:

apt-get update
apt-get install qemu kvm libvirt-bin virt-top virtinst
apt-get install openssh-server screen bridge-utils

Optionally, install some useful software as well:

apt-get install vim zsh screen moreutils colordiff monkeytail rsync rdiff rdiff-backup pwgen lsof iotop
apt-get install tcpdump traceroute
apt-get install ntp                 # only on main host
apt-get install munin munin-node    # nice load graphs

STEP 2. Prepare user to operate virtual instanses (general)

While "root" will work, I recommend strongly to execute daily tasks in ordinary user account, for security reasons. It may be user created by debian installer.

Allow user to operate "system" (common) instanses. While libvirt allows each user to create its own instanses, we will not use this feature for the sake of simplicity.

adduser frank
adduser frank libvirt
echo "VIRSH_DEFAULT_CONNECT_URI=qemu:///system" >> /etc/environment

And relogin frank to take effect.

STEP 3. Replacing physical eth0 with bridge device on host machine

WARN: on this stage you may lost connection (including, but not limited to ssh) to your machine. If you have neither local console access, neither IP-KVM, neither confidence of success, skip this and set up port forwardings like described below.

If you do not want to configure firewall and/or routing rules manually on host for each virtual machine external IP, do following:

Check if bridge-utils are installed and available:

brctl addbr br0

Then replace eth0 with br0 in /etc/network/interfaces, "allow-hotplug eth0" with "auto br0" and add:

...br0...
	bridge_ports eth0
	bridge_stp on
	bridge_maxwait 0

iface eth0

So, if your /etc/network/interfaces looks like:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug eth0
iface eth0 inet static
	address 172.20.60.24
	netmask 255.255.255.0
	network 172.20.60.0
	broadcast 172.20.60.255
	gateway 172.20.60.1

Change it to:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto br0
iface br0 inet static
	address 172.20.60.24
	netmask 255.255.255.0
	network 172.20.60.0
	broadcast 172.20.60.255
	gateway 172.20.60.1
	bridge_ports eth0
	bridge_stp on
	bridge_maxwait 0
	bridge_fd 5

Better to replace this in one step with possible rollback. Backup your /etc/network/interfaces and prepare new "interfaces":

cd /etc/network
cp -a interfaces interfaces.eth0
cp -a interfaces interfaces.br0
editor interfaces.br0       # or upload new config

Prepare the script to switch interfaces "switch-to-br0.sh", and execute it inside "screen -xR" (to survive ssh disconnect):

screen -xR
./switch-to-br0.sh

Rarely, SSH connect may hang, so you will need to reconnect after 10-30 seconds and execute successfull reconnect:

touch /etc/interfaces/ok

to stop script above from rolling back interfaces settings.

Reboot host machine just to be sure.

STEP 3. Configuring network interfaces in libvirt

Check if you connected to right host. Execute in virsh shell:

net-list --all
net-edit default

I recommend to shrink IP range for DHCP to somethink like 128-254. It allows you to set up fixed IP on virtual machines without conflicts.

It is good idea to autostart network after host reboot:

net-autostart default

Note that it is right to previously configured bridge interface br0 does not appear here.

Also, you can forward some connections to the instance not having external IP

iptables -A PREROUTING -t nat -d 172.20.60.24 -i br0 -p tcp -m tcp --dport 3389 -j DNAT --to-destination 192.168.122.7:3389
iptables -A PREROUTING -t nat -d 172.20.60.24 -i br0 -p tcp -m tcp --dport 3390 -j DNAT --to-destination 192.168.122.8:3389
iptables -A PREROUTING -t nat -d 172.20.60.24 -i br0 -p tcp -m tcp --dport 4444 -j DNAT --to-destination 192.168.122.9:3389

Note that source and destination port do not needed to be equal.

To make these "prerouting" rules work, execute

iptables -I FORWARD -i br0 -o virbr0 -j ACCEPT

STEP 5. Configuring storage pools

I prefer to have two separate pools, for hard disk images and for OS iso images respectively. Hard disk images stored in /var/lib/libvirt/images by default.

Along the way commands "*-list --all" and "*-dumpxml" provided just for reference, they not needed to actually create or modify things. :-)

To change path, execute in virsh shell

pool-list --all
# just "stops" pool, nothing harmful occur
pool-destroy default
pool-list --all
# opens xml config in editor, save and exit
pool-edit default
# start previously defined pool
pool-create default
pool-list --all

To add new storage pool:

pool-define-as iso dir --target /srv/libvirt/iso
pool-list --all
pool-dumpxml iso
pool-start iso
pool-autostart iso

To disable autostart:

pool-autostart --disable iso

To refresh pool after manual add some images not using virsh:

pool-refresh default
pool-refresh iso

STEP 6. Defining virtual machines

First, create hard disk image:

vol-create-as default disk1.img 4G --allocation 0 --format raw
vol-create-as default disk2.img 4G --allocation 0 --format raw
vol-list default

Second, copy and edit provided template "vds-template.xml" and define virtual machine from resulting xml.

Changing the name of virtual machine is recommended. Otherwise, after "renaming" by editing xml you'll end up with two machines.

define vds-template.xml
edit machine1

STEP 7. Managing virtual machines

help
list --all
start machine1
reboot machine1
shutdown machine1
destroy machine1
edit machine1

Changes to running guest usually take effect after "destroy", then "start", not after just guest reboot.

Do not forget to set autostart on machines:

autostart machine1
autostart --disable machine1

To rename guest:

Stop guest ("destroy machine1").
Edit its xml ("edit machine1"): change name and delete uuid.
Delete old named guest: "undefine machine1".
Start guest.

When editing existing machines feel free to delete "address" entries to avoid inconsistency in its values.

Futher reading

STEP X. Client config (linux gui)

Install virtual machine manager GUI on DESKTOP (no version for windows exists):

apt-get install virt-manager

Try to connect:

File -> Add Connection -> Connection: Remote tunnel over SSH, enter hostname or IP

New host should appear in list in main window.

Right click -> Details -> Storage

Optionally add Storage Pool with name "iso" and type "dir: Filesystem Directory" pointed at directory where OS install images are located.

Right click -> New

If virt-manager doesn't show warning "KVM is not available," virtualization on host machine is configured correctly.