Kernel-based Virtual Machine (KVM) is the method to run multiple guest operating systems on one RouterOS host. KVM can be used only on x86 machines that have CPU with virtualization support .
KVM requires Intel VT-x or AMD-V CPU virtualization support. Here you can find a list of supported CPUs, for more detailed information look on vendor's web site.
Each guest requires at least 16 MB of RAM and sufficient storage space on image file. Once image file have been created, its size cannot be increased.
KVM support in RouterOS is enabled if kvm package is installed.
Where it can be used?
Virtual Router is useful to allow clients or lower-privilege users access their own 'router' and adjust configure as they like without the need for a second hardware.
For example; a WISP can create a virtual router for the clients ethernet port allowing them to define their own firewall settings, while leaving the WISP's wireless settings untouched.
Another useful method is to run guest OS that supports functionality which is not available in RouterOS, for example, Intrusion detection (SNORT), Asterisk or Squid web proxy.
It can also be used as test environment. it is possible to create virtual network within one x86 machine very similar to real network and test how RouterOS behaves before implementing the setup in your production network.
Creating KVM Guest
Before creating KVM guest we need image file. RouterOS has built in commands to make and modify RouterOS image easily without external tools.
/kvm make-routeros-image file-name=ros1.img file-size=128
We can proceed with Guest configuration when disk image is created.
/kvm add name=ROS memory=128MiB cpu-count=2 disabled=no disk-images=hda:ros1.img \ initrd="" kernel="" kernel-cmdline="console=ttyS0"
As you noticed initrd and kernel properties are empty, which means that hosts kernel and initrd is used.
For example, to add guest without SMP support we can explicitly set initrd and kernel:
/kvm add name=ROS memory=128MiB cpu-count=2 disabled=no disk-images=hda:ros1.img \ initrd=/boot/initrd.rgz kernel=/boot/vmlinuz kernel-cmdline="console=ttyS0"
Note: Leaving initrd and kernel properties empty is dangerous if Host and Guest will be running different RouterOS versions . Guests other than RouterOS also can break if you leave these values empty.
KVM Guest when created is not automatically started. We must start it manually
[admin@proxy] /kvm> start ROS;
[admin@proxy] /kvm> print Flags: X - disabled 0 name="ROS" cpu-count=2 memory=128MiB disk-images=hda:ros1.img kernel="/boot/vmlinuz" kernel-cmdline="" initrd="/boot/initrd.rgz" vnc-server=0.0.0.0:0 snapshot=no state=running [admin@proxy] /kvm>
Lets add to our previously created Virtual Router one interface.
[admin@proxy] /kvm interface> add virtual-machine=ROS type=dynamic [admin@proxy] /kvm interface> print Flags: X - disabled, A - active # VIRTUAL-MACHINE INTERFACE TYPE VM-MAC-ADDRESS 0 ROS dynamic 02:D9:52:31:11:CC [admin@proxy] /kvm interface>
In this case dynamic type is used which creates dynamic virtual interface on the host:
[admin@proxy] /interface virtual-ethernet> print Flags: D - dynamic, X - disabled, R - running # NAME MTU ARP MAC-ADDRESS 0 D R tap1 1500 enabled 02:3F:9F:AE:10:34 [admin@proxy] /interface virtual-ethernet>
Note: Add and remove interfaces only when KVM guest is shut-down, stopped or disabled. Making changes to running guest may lead to host system crash.
If mac addresses are not specified when creating virtual interfaces, addresses are generated automatically. Generate MAC addresses will be in form of 02:XX:XX:XX:XX:XX. For static interfaces this address will not change during use of guest, for dynamic interface will change every time dynamic interface is created.
More information about virtual interfaces are in virtual-ethernet manual
Connecting to the virtual machine
There are two ways how to connect to KVM Guest:
- virtual console;
To connect using console:
[admin@proxy] /kvm> console ROS
You will see your newly added virtual interface here:
[admin@mr0] > interface print Flags: D - dynamic, X - disabled, R - running, S - slave # NAME TYPE MTU 0 R ether1 ether 1500
To disconnect from the metarouter virtual machine console, hit CTRL + A and then Q to Quit back to your Host console (if you are using minicom, hit CTRL + A twice):
[admin@MikroTik] > [Q - quit connection] [B - send break] [A - send Ctrl-A prefix] [R - autoconfigure rate] Q Welcome back!
Before connecting with VNC client guest needs some configuration changes.
[admin@proxy] /kvm> print Flags: X - disabled 0 name="ROS" cpu-count=2 memory=128MiB disk-images=hda:ros1.img kernel="/boot/vmlinuz" kernel-cmdline="" initrd="/boot/initrd.rgz" vnc-server=0.0.0.0:0 snapshot=no state=running [admin@proxy] /kvm> shut-down 0 [admin@proxy] /kvm> set 0 vnc-server=10.5.100.99:1 [admin@proxy] /kvm> start 0
[admin@proxy] /kvm> print Flags: X - disabled 0 name="ROS" cpu-count=2 memory=128MiB disk-images=hda:ros1.img kernel="/boot/vmlinuz" kernel-cmdline="" initrd="/boot/initrd.rgz" vnc-server=10.5.100.99:1 snapshot=no state=running [admin@proxy] /kvm>
VNC servers address in this case is the address on the host reachable from remote locations. Address is followed by screen number.
Now we can try to connect from remote location:
mrz@bumba:/$ vncviewer 10.5.100.99:1
Configuring a virtual network
Right now you saw that the virtual interface is visible in the Host Interfaces menu as tap1 and also in the guest interfaces menu as ether1. You can add an IP address on both interfaces, and set up networking. Creating a bridge between the virtual interface and a physical interface allows traffic to pass.
As an example lets make three virtual routers connected to each other on the same broadcast domain. File:Kvm-test-bcast.png
Create images and guests:
/kvm make-routeros-image file-name=R1.img file-size=64 make-routeros-image file-name=R2.img file-size=64 make-routeros-image file-name=R3.img file-size=64 add name=R1 disk-image=hda:R1.img add name=R2 disk-image=hda:R2.img add name=R3 disk-image=hda:R3.img
Create a bridge interface which will simulate broadcast domain and add virtual interfaces:
/interface bridge add name=kvm_bridge /kvm interface add virtual-machine=R1 type=dynamic dynamic-bridge=kvm_bridge add virtual-machine=R2 type=dynamic dynamic-bridge=kvm_bridge add virtual-machine=R3 type=dynamic dynamic-bridge=kvm_bridge
Now we can start virtual machines and verify if dynamic interfaces are created:
[admin@proxy] /kvm> start [find]
[admin@proxy] > /interface virtual-ethernet print Flags: D - dynamic, X - disabled, R - running # NAME MTU ARP MAC-ADDRESS 0 D R tap2 1500 enabled 02:20:94:67:D6:D5 1 D R tap3 1500 enabled 02:95:EE:EA:43:FF 2 D R tap4 1500 enabled 02:05:7E:4B:86:F9 [admin@proxy] > /interface bridge port print Flags: X - disabled, I - inactive, D - dynamic # INTERFACE BRIDGE PRIORITY PATH-COST HORIZON 0 D tap2 kvm_bridge 0x80 10 none 1 D tap3 kvm_bridge 0x80 10 none 2 D tap4 kvm_bridge 0x80 10 none [admin@proxy] >
Now we can connect with console to each of guests and set up ip addresses from the same network and verify reachability. R1
[admin@proxy] > /kvm console R1 [Ctrl-A is the prefix key] MikroTik 5.0rc8 MikroTik Login: admin Password: [admin@MikroTik] > /ip address add address=192.168.1.1/24 interface=ether1
<pre> [admin@proxy] > /kvm console R2 [Ctrl-A is the prefix key] MikroTik 5.0rc8 MikroTik Login: admin Password: [admin@MikroTik] > /ip address add address=192.168.1.2/24 interface=ether1
<pre> [admin@proxy] > /kvm console R1 [Ctrl-A is the prefix key] MikroTik 5.0rc8 MikroTik Login: admin Password: [admin@MikroTik] > /ip address add address=192.168.1.3/24 interface=ether1 [admin@MikroTik] > /ping 192.168.1.1 HOST SIZE TTL TIME STATUS 192.168.1.1 56 64 11ms 192.168.1.1 56 64 2ms sent=2 received=2 packet-loss=0% min-rtt=2ms avg-rtt=6ms max-rtt=11ms [admin@MikroTik] > /ping 192.168.1.2 HOST SIZE TTL TIME STATUS 192.168.1.2 56 64 12ms sent=1 received=1 packet-loss=0% min-rtt=12ms avg-rtt=12ms max-rtt=12ms
Removing KVM guest
KVM guest has two parts in RouterOS - configuration (kvm, virtual-ethernet, /kvm interface) and image file (/file). If image file is removed, but KVM guest is still running, then file will be removed from file menu but still exist until guest is shut-down or disabled, at that moment file will be removed and storage space returned to available storage on the router.
Information useful for running KVM guests
When host is shutting down each guest receives shut-down notification and are give 10 seconds to shut down. After time-out value is reached, guests are killed.
Host and guest update
When new version of RouterOS is updated to host system and you have RouterOS guest with initrd and kernel fields empty, it is good practice to update guest first (even it it does not boot up at current host versions. Then update host and see if guests are running. After guest update incompatibilities between host kernel and guest drivers might prevent guest from booting up properly.
KVM Guest Properties
To add new KVM guest you will have to issue command add under /kvm menu with attributes as follows:
|comment (text, default: ')||to add simple text description of the KVM guest|
|cpu-count (1 .. 32, default: 1)||available count of processing cores for guest. Allowed values are [1..32]|
|disabled (yes | no, default: no)||to set guest state after creation, values: yes or no|
|disk-images ( list of images used in guest)||list of image assignment to drives for guest OS. If type will be set to cdrom then guest will automatically boot from that, instead of any other drive configured in this field. It can be single drive specified
or it can be comma seperated list:
|initrd (path)||path to initrd file, can be left empty if running RouterOS as guest|
|kernel (path)||path to kernel image file, if using RouterOS image created on host this field can be left empty|
|kernel-cmdline (text)||parameters that are passed to kernel, it is space separated string.|
|memory (integer default:32)||to set up amount of memory that is available to KVM guest|
|name (text)||name of KVM guest that it will be accessible though the system|
|snapshot (yes | no)||will try to run virtual machine with image file in read-only mode.|
|vnc-server-address (IP address )||address to bind VNC server port that will connect to guest virtual screen. If left empty it will bind to all IP addresses. If address set is not ready at the moment when guest is started then system will automatically attempt to start guest for the next 20 seconds. If IP address to bind VNC does not become available in that time automatic start of guest will fail and guest will not be started. IP address is considered unavailable if either address or interface address is assigned to is invalid or does not exist.|
|vnc-server-display (number (0..99) default:0)||will try to run virtual machine with image file in read-only|
|copy-from (number)||use configuration from already existing KVM guest|
Warning: vnc-server attribute has been changed since RouterOS 5.0. in older versions instead of vnc-server-address and vnc-server-display was used combine attribute named vnc-server <IP address>:<display number>
Note: If start of guest failed for the first time, then next 20 seconds KVM will attempt to start guest. After 20 seconds it will fail and guest will stay in stopped state.
States of KVM guest
This field is read-only and is set by RouterOS. These are possible values that can be set:
- stopped - KVM guest is not running, either successful shut-down or disabled.
- stopping - KVM guest is shutting down
- starting - KVM guest is starting
- running - KVM guest has started successfully and is executing guest operating system
- restarting - KVM guest is reloading its guest operating system
- failed - KVM guest has encountered an error and is not operational.
- image-busy - image file set in configuration is already in use by other KVM guest entry
- no-kernel-or-initrd - initrd or kernel was not found in files set in configuration, mentioned files could not be found or no values in those fields where set
- no-disk-image - either disk image was not found or disk image was not set in configuration.
- kernel-extract-failed - when in guest configuration field kernel is left empty and and KVM cannot extract kernel from image file supplied
- vnc-cant-bind - vnc server for guest cannot bind to setting specified in vnc-server-address and/or vnc-server-display
Sub-menu allows to manage KVM guests on RouterOS host.
|add||Create new KVM guest entry|
|comment||Set comment for KVM guest entry|
|console||to connect to KVM guest console display|
|continue||resume KVM guest if it was paused|
|disable||change global state of KVM guest. If enabled KVM guest will be started when RouterOS boots. KVM guest cannot change|
|edit||edit selected value of KVM guest entry|
|enable||change KVM guest global state to enable operation of KVM guest. If guest where disabled before - KVM guest is automatically started.|
|export||Print or save an export script that can be used to restore configuration of current sub-menu, KVM guest configuration, image files will not be saved|
|find||Find items by value|
|get||Gets value of item's property|
|make-routeros-image||creates RouterOS image from current installation installed on the router with no configuration. It is advised to create Image file larger than minimal, so you are able to upload new package files and upgrade/update RouterOS installation. Also, all the additional files created in KVM guest will be stored in file image. This image file is not connected to host RouterOS and user is able to run different RouterOS versions on host and guest. This command will create RAW image file containing RouterOS installation.
|pause||suspend operation of KVM guest|
|Print values of item properties|
|reboot||issue ACPI shut-down command to KVM guest, if guest does not support ACPI, command have no effect. After KVM guest is shut-downed it will be automatically started by host when shut down is complete.|
|reconfigure-routeros-image||sets up default configuration for RouterOS image.
|set||Change item properties|
|shut-down||issue ACPI shut-down command to KVM guest, if guest does not support ACPI, command have no effect.|
|start||to start KVM guest|
|comment (text)||description of interface|
|disabled (yes|no, default: no)||state of interface after creation|
|host-mac-address (MAC Address, default: generated)||MAC address of virtual interface that host will use|
|model (virto | e1000 | pcnet, default: virtio)||mode of virtual interface. Available options are:
|vm-mac-address (MAC Address, default: generated)||MAC address of virtual interface that guest will use|
|copy-from (number)||use configuration from existing virtual interface|
|dynamic-bridge (interface name, default: none)||if set, dynamic interface will be automatically added as port to bridge interface|
|interface||is set for static interface, to assign it to already created virtual-ethernet interface|
|type (dynamic | static, default: static)||to set if interface is either static or dynamic.
|virtual-machine (KVM machine name, must be set)||name of virtual machine this interface will be assigned to|