Manual:KVM

Applies to RouterOS: v4.3+ on x86

Overview

Kernel-based Virtual Machine (KVM) is the method to run multiple guest operating systems on one RouterOS host. KVM can be used only on x86 machines that have CPU with virtualization support .

Requirements

KVM requires Intel VT-x or AMD-V CPU virtualization support. Here you can find a list of supported CPUs, for more detailed information look on vendor's web site.

Each guest requires at least 16 MB of RAM and sufficient storage space on image file. Once image file have been created, its size cannot be increased.

KVM support in RouterOS is enabled if kvm package is installed.

Where it can be used?

Virtual Router is useful to allow clients or lower-privilege users access their own 'router' and adjust configure as they like without the need for a second hardware.

For example; a WISP can create a virtual router for the clients ethernet port allowing them to define their own firewall settings, while leaving the WISP's wireless settings untouched.

Another useful method is to run guest OS that supports functionality which is not available in RouterOS, for example, Intrusion detection (SNORT), Asterisk or Squid web proxy.

It can also be used as test environment. it is possible to create virtual network within one x86 machine very similar to real network and test how RouterOS behaves before implementing the setup in your production network.

Creating KVM Guest

Before creating KVM guest we need image file. RouterOS has built in commands to make and modify RouterOS image easily without external tools.

/kvm make-routeros-image file-name=ros1.img file-size=128

We can proceed with Guest configuration when disk image is created.

/kvm add name=ROS memory=128MiB cpu-count=2 disabled=no disk-images=hda:ros1.img \
    initrd="" kernel="" kernel-cmdline="console=ttyS0"

As you noticed initrd and kernel properties are empty, which means that hosts kernel and initrd is used.

For example, to add guest without SMP support we can explicitly set initrd and kernel:

/kvm add name=ROS memory=128MiB cpu-count=2 disabled=no disk-images=hda:ros1.img \
    initrd=/boot/initrd.rgz kernel=/boot/vmlinuz kernel-cmdline="console=ttyS0"

Note: Leaving initrd and kernel properties empty is dangerous if Host and Guest will be running different RouterOS versions . Guests other than RouterOS also can break if you leave these values empty.

KVM Guest when created is not automatically started. We must start it manually

[admin@proxy] /kvm> start ROS; 

[admin@proxy] /kvm> print 
Flags: X - disabled 
 0   name="ROS" cpu-count=2 memory=128MiB disk-images=hda:ros1.img kernel="/boot/vmlinuz" 
      kernel-cmdline="" initrd="/boot/initrd.rgz" vnc-server=0.0.0.0:0 snapshot=no  state=running 
[admin@proxy] /kvm> 

Adding Interfaces

Lets add to our previously created Virtual Router one interface.

[admin@proxy] /kvm interface> add virtual-machine=ROS type=dynamic 
[admin@proxy] /kvm interface> print 
Flags: X - disabled, A - active 
 #   VIRTUAL-MACHINE            INTERFACE                  TYPE        VM-MAC-ADDRESS   
 0   ROS                                                   dynamic     02:D9:52:31:11:CC
[admin@proxy] /kvm interface>

In this case dynamic type is used which creates dynamic virtual interface on the host:

[admin@proxy] /interface virtual-ethernet> print 
Flags: D - dynamic, X - disabled, R - running 
 #     NAME                             MTU   ARP        MAC-ADDRESS      
 0 D R tap1                             1500  enabled    02:3F:9F:AE:10:34
[admin@proxy] /interface virtual-ethernet> 

Note: Add and remove interfaces only when KVM guest is shut-down, stopped or disabled. Making changes to running guest may lead to host system crash.

If mac addresses are not specified when creating virtual interfaces, addresses are generated automatically. Generate MAC addresses will be in form of 02:XX:XX:XX:XX:XX. For static interfaces this address will not change during use of guest, for dynamic interface will change every time dynamic interface is created.

More information about virtual interfaces are in virtual-ethernet manual

Connecting to the virtual machine

There are two ways how to connect to KVM Guest:

• virtual console;
• vnc.

Console

To connect using console:

[admin@proxy] /kvm> console ROS

You will see your newly added virtual interface here:

 [admin@mr0] > interface print 
  Flags: D - dynamic, X - disabled, R - running, S - slave 
  #     NAME                                              TYPE             MTU  
  0  R  ether1                                            ether            1500

To disconnect from the metarouter virtual machine console, hit CTRL + A and then Q to Quit back to your Host console (if you are using minicom, hit CTRL + A twice):

 [admin@MikroTik] >
 [Q - quit connection]      [B - send break]
 [A - send Ctrl-A prefix]   [R - autoconfigure rate]
  
  Q
  
  Welcome back!

VNC

Before connecting with VNC client guest needs some configuration changes.

[admin@proxy] /kvm> print 
Flags: X - disabled 
 0   name="ROS" cpu-count=2 memory=128MiB disk-images=hda:ros1.img kernel="/boot/vmlinuz" 
      kernel-cmdline="" initrd="/boot/initrd.rgz" vnc-server=0.0.0.0:0 snapshot=no state=running 
[admin@proxy] /kvm> shut-down 0
[admin@proxy] /kvm> set 0 vnc-server=10.5.100.99:1
[admin@proxy] /kvm> start 0

[admin@proxy] /kvm> print 
Flags: X - disabled 
 0   name="ROS" cpu-count=2 memory=128MiB disk-images=hda:ros1.img kernel="/boot/vmlinuz" 
      kernel-cmdline="" initrd="/boot/initrd.rgz" vnc-server=10.5.100.99:1 snapshot=no
      state=running 
[admin@proxy] /kvm> 

VNC servers address in this case is the address on the host reachable from remote locations. Address is followed by screen number.

Now we can try to connect from remote location:

mrz@bumba:/$ vncviewer 10.5.100.99:1

Configuring a virtual network

Right now you saw that the virtual interface is visible in the Host Interfaces menu as tap1 and also in the guest interfaces menu as ether1. You can add an IP address on both interfaces, and set up networking. Creating a bridge between the virtual interface and a physical interface allows traffic to pass.

As an example lets make three virtual routers connected to each other on the same broadcast domain. File:Kvm-test-bcast.png

Create images and guests:

/kvm 
make-routeros-image file-name=R1.img file-size=64
make-routeros-image file-name=R2.img file-size=64
make-routeros-image file-name=R3.img file-size=64

add name=R1 disk-image=hda:R1.img
add name=R2 disk-image=hda:R2.img
add name=R3 disk-image=hda:R3.img

Create a bridge interface which will simulate broadcast domain and add virtual interfaces:

/interface bridge
add name=kvm_bridge

/kvm interface
add virtual-machine=R1 type=dynamic dynamic-bridge=kvm_bridge
add virtual-machine=R2 type=dynamic dynamic-bridge=kvm_bridge
add virtual-machine=R3 type=dynamic dynamic-bridge=kvm_bridge

Now we can start virtual machines and verify if dynamic interfaces are created:

[admin@proxy] /kvm> start [find]

[admin@proxy] > /interface virtual-ethernet print 
Flags: D - dynamic, X - disabled, R - running 
 #     NAME                             MTU   ARP        MAC-ADDRESS      
 0 D R tap2                             1500  enabled    02:20:94:67:D6:D5
 1 D R tap3                             1500  enabled    02:95:EE:EA:43:FF
 2 D R tap4                             1500  enabled    02:05:7E:4B:86:F9
[admin@proxy] > /interface bridge port print
Flags: X - disabled, I - inactive, D - dynamic 
 #    INTERFACE                BRIDGE                    PRIORITY PATH-COST  HORIZON   
 0  D tap2                     kvm_bridge                0x80     10         none      
 1  D tap3                     kvm_bridge                0x80     10         none      
 2  D tap4                     kvm_bridge                0x80     10         none      
[admin@proxy] > 

Now we can connect with console to each of guests and set up ip addresses from the same network and verify reachability. R1

[admin@proxy] > /kvm console R1

[Ctrl-A is the prefix key]

MikroTik 5.0rc8
MikroTik Login: admin
Password:

 [admin@MikroTik] > /ip address add address=192.168.1.1/24 interface=ether1 

R2

<pre>
[admin@proxy] > /kvm console R2

[Ctrl-A is the prefix key]

MikroTik 5.0rc8
MikroTik Login: admin
Password:

 [admin@MikroTik] > /ip address add address=192.168.1.2/24 interface=ether1 

R3

<pre>
[admin@proxy] > /kvm console R1

[Ctrl-A is the prefix key]

MikroTik 5.0rc8
MikroTik Login: admin
Password:

[admin@MikroTik] > /ip address add address=192.168.1.3/24 interface=ether1 
[admin@MikroTik] > /ping 192.168.1.1
HOST                                    SIZE  TTL TIME  STATUS
192.168.1.1                             56    64  11ms 
192.168.1.1                             56    64  2ms  
    sent=2 received=2 packet-loss=0% min-rtt=2ms avg-rtt=6ms max-rtt=11ms 

[admin@MikroTik] > /ping 192.168.1.2
HOST                                    SIZE  TTL TIME  STATUS    
192.168.1.2                             56    64  12ms 
    sent=1 received=1 packet-loss=0% min-rtt=12ms avg-rtt=12ms max-rtt=12ms 

Removing KVM guest

KVM guest has two parts in RouterOS - configuration (kvm, virtual-ethernet, /kvm interface) and image file (/file). If image file is removed, but KVM guest is still running, then file will be removed from file menu but still exist until guest is shut-down or disabled, at that moment file will be removed and storage space returned to available storage on the router.

Additional information

Information useful for running KVM guests

Host shutdown

When host is shutting down each guest receives shut-down notification and are give 10 seconds to shut down. After time-out value is reached, guests are killed.

Host and guest update

When new version of RouterOS is updated to host system and you have RouterOS guest with initrd and kernel fields empty, it is good practice to update guest first (even it it does not boot up at current host versions. Then update host and see if guests are running. After guest update incompatibilities between host kernel and guest drivers might prevent guest from booting up properly.

Reference

General

Sub-menu: /kvm

KVM Guest Properties

To add new KVM guest you will have to issue command add under /kvm menu with attributes as follows:

Property	Desciption
comment (text, default: ')	to add simple text description of the KVM guest
cpu-count (1 .. 32, default: 1)	available count of processing cores for guest. Allowed values are [1..32]
disabled (yes | no, default: no)	to set guest state after creation, values: yes or no
disk-images ( list of images used in guest)	list of image assignment to drives for guest OS. If type will be set to cdrom then guest will automatically boot from that, instead of any other drive configured in this field. It can be single drive specified disk-images=hda:ros.img or it can be comma seperated list: disk-images=hda:system.img,hdb:swap.img
initrd (path)	path to initrd file, can be left empty if running RouterOS as guest
kernel (path)	path to kernel image file, if using RouterOS image created on host this field can be left empty
kernel-cmdline (text)	parameters that are passed to kernel, it is space separated string.
memory (integer default:32)	to set up amount of memory that is available to KVM guest
name (text)	name of KVM guest that it will be accessible though the system
snapshot (yes | no)	will try to run virtual machine with image file in read-only mode.
vnc-server-address (IP address )	address to bind VNC server port that will connect to guest virtual screen. If left empty it will bind to all IP addresses. If address set is not ready at the moment when guest is started then system will automatically attempt to start guest for the next 20 seconds. If IP address to bind VNC does not become available in that time automatic start of guest will fail and guest will not be started. IP address is considered unavailable if either address or interface address is assigned to is invalid or does not exist.
vnc-server-display (number (0..99) default:0)	will try to run virtual machine with image file in read-only
copy-from (number)	use configuration from already existing KVM guest

Warning: vnc-server attribute has been changed since RouterOS 5.0. in older versions instead of vnc-server-address and vnc-server-display was used combine attribute named vnc-server <IP address>:<display number>

Note: If start of guest failed for the first time, then next 20 seconds KVM will attempt to start guest. After 20 seconds it will fail and guest will stay in stopped state.

States of KVM guest

This field is read-only and is set by RouterOS. These are possible values that can be set:

• stopped - KVM guest is not running, either successful shut-down or disabled.
• stopping - KVM guest is shutting down
• starting - KVM guest is starting
• running - KVM guest has started successfully and is executing guest operating system
• restarting - KVM guest is reloading its guest operating system
• failed - KVM guest has encountered an error and is not operational.
• image-busy - image file set in configuration is already in use by other KVM guest entry
• no-kernel-or-initrd - initrd or kernel was not found in files set in configuration, mentioned files could not be found or no values in those fields where set
• no-disk-image - either disk image was not found or disk image was not set in configuration.
• kernel-extract-failed - when in guest configuration field kernel is left empty and and KVM cannot extract kernel from image file supplied
• vnc-cant-bind - vnc server for guest cannot bind to setting specified in vnc-server-address and/or vnc-server-display

KVM commands

Sub-menu allows to manage KVM guests on RouterOS host.

Command	Desciption
add	Create new KVM guest entry
comment	Set comment for KVM guest entry
console	to connect to KVM guest console display
continue	resume KVM guest if it was paused
disable	change global state of KVM guest. If enabled KVM guest will be started when RouterOS boots. KVM guest cannot change
edit	edit selected value of KVM guest entry
enable	change KVM guest global state to enable operation of KVM guest. If guest where disabled before - KVM guest is automatically started.
export	Print or save an export script that can be used to restore configuration of current sub-menu, KVM guest configuration, image files will not be saved
find	Find items by value
get	Gets value of item's property
make-routeros-image	creates RouterOS image from current installation installed on the router with no configuration. It is advised to create Image file larger than minimal, so you are able to upload new package files and upgrade/update RouterOS installation. Also, all the additional files created in KVM guest will be stored in file image. This image file is not connected to host RouterOS and user is able to run different RouterOS versions on host and guest. This command will create RAW image file containing RouterOS installation. parameters: file-name - name of ROS image file; file-size - image size in Meba Bytes; configuration-script - file name where configuration script is located;
pause	suspend operation of KVM guest
print	Print values of item properties
reboot	issue ACPI shut-down command to KVM guest, if guest does not support ACPI, command have no effect. After KVM guest is shut-downed it will be automatically started by host when shut down is complete.
reconfigure-routeros-image	sets up default configuration for RouterOS image. Parameters: file-name - name of ROS image to be reconfigured; configuraton-script - file name where configuration script is located; configuration-string - string containing ROS commands to be configured on ROS image.
remove	Remove item
set	Change item properties
shut-down	issue ACPI shut-down command to KVM guest, if guest does not support ACPI, command have no effect.
start	to start KVM guest

Interface

Sub-menu: /kvm interface

Property	Desciption
comment (text)	description of interface
disabled (yes|no, default: no)	state of interface after creation
host-mac-address (MAC Address, default: generated)	MAC address of virtual interface that host will use
model (virto | e1000 | pcnet, default: virtio)	mode of virtual interface. Available options are: virtio - default value. Fastest available option, should be chosen if no other problems are encountered e1000 - emulates card that uses e1000 driver. This option where added for compatibility with some guest operating systems that where not able to communicate with host RouterOS if virtio interface model where used. pcnet - emulates card that uses pcnet driver. This option where added for compatibility with some guest operating systems that where not able to communicate with host RouterOS if virtio interface model where used.
vm-mac-address (MAC Address, default: generated)	MAC address of virtual interface that guest will use
copy-from (number)	use configuration from existing virtual interface
dynamic-bridge (interface name, default: none)	if set, dynamic interface will be automatically added as port to bridge interface
interface	is set for static interface, to assign it to already created virtual-ethernet interface
type (dynamic | static, default: static)	to set if interface is either static or dynamic. dynamic interface will add virtual-ethernet automatically when virtual machine starts. static interface have to have created virtual-ethernet interface at the time of creation of the entry.
virtual-machine (KVM machine name, must be set)	name of virtual machine this interface will be assigned to

See also

• Wikipedia about virtualization support on x86 CPUs
• Virtual Ethernet
• MetaROUTER