Manual:KVM
Applies to RouterOS: v4.3+ on x86
Overview
Kernel-based Virtual Machine (KVM) is the method to run multiple guest operating systems on one RouterOS host. KVM can be used only on x86 machines that have CPU with virtualization support .
Requirements
KVM requires Intel VT-x or AMD-V CPU virtualization support. Here you can find a list of supported CPUs, for more detailed information look on vendor's web site.
Each guest requires at least 16 MB of RAM and sufficient storage space on image file. Once image file have been created, its size cannot be increased.
KVM support in RouterOS is enabled if kvm package is installed.
Where it can be used?
Virtual Router is useful to allow clients or lower-privilege users access their own 'router' and adjust configure as they like without the need for a second hardware.
For example; a WISP can create a virtual router for the clients ethernet port allowing them to define their own firewall settings, while leaving the WISP's wireless settings untouched.
Another useful method is to run guest OS that supports functionality which is not available in RouterOS, for example, Intrusion detection (SNORT), Asterisk or Squid web proxy.
It can also be used as test environment. it is possible to create virtual network within one x86 machine very similar to real network and test how RouterOS behaves before implementing the setup in your production network.
Creating KVM Guest
Before creating KVM guest we need image file. RouterOS has built in commands to make and modify RouterOS image easily without external tools.
/kvm make-routeros-image file-name=ros1.img file-size=128
We can proceed with Guest configuration when disk image is created.
/kvm add name=ROS memory=128MiB cpu-count=2 disabled=no disk-images=hda:ros1.img \
initrd="" kernel="" kernel-cmdline="console=ttyS0"
As you noticed initrd and kernel properties are empty, which means that hosts kernel and initrd is used.
For example, to add guest without SMP support we can explicitly set initrd and kernel:
/kvm add name=ROS memory=128MiB cpu-count=2 disabled=no disk-images=hda:ros1.img \
initrd=/boot/initrd.rgz kernel=/boot/vmlinuz kernel-cmdline="console=ttyS0"
Note: Leaving initrd and kernel properties empty is dangerous if Host and Guest will be running different RouterOS versions . Guests other than RouterOS also can break if you leave these values empty.
KVM Guest when created is not automatically started. We must start it manually
[admin@proxy] /kvm> start ROS;
[admin@proxy] /kvm> print
Flags: X - disabled
0 name="ROS" cpu-count=2 memory=128MiB disk-images=hda:ros1.img kernel="/boot/vmlinuz"
kernel-cmdline="" initrd="/boot/initrd.rgz" vnc-server=0.0.0.0:0 snapshot=no state=running
[admin@proxy] /kvm>
Adding Interfaces
Lets add to our previously created Virtual Router one interface.
[admin@proxy] /kvm interface> add virtual-machine=ROS type=dynamic [admin@proxy] /kvm interface> print Flags: X - disabled, A - active # VIRTUAL-MACHINE INTERFACE TYPE VM-MAC-ADDRESS 0 ROS dynamic 02:D9:52:31:11:CC [admin@proxy] /kvm interface>
In this case dynamic type is used which creates dynamic virtual interface on the host:
[admin@proxy] /interface virtual-ethernet> print Flags: D - dynamic, X - disabled, R - running # NAME MTU ARP MAC-ADDRESS 0 D R tap1 1500 enabled 02:3F:9F:AE:10:34 [admin@proxy] /interface virtual-ethernet>
Note: Add and remove interfaces only when KVM guest is shut-down, stopped or disabled. Making changes to running guest may lead to host system crash.
If mac addresses are not specified when creating virtual interfaces, addresses are generated automatically. Generate MAC addresses will be in form of 02:XX:XX:XX:XX:XX. For static interfaces this address will not change during use of guest, for dynamic interface will change every time dynamic interface is created.
More information about virtual interfaces are in virtual-ethernet manual
Connecting to the virtual machine
There are two ways how to connect to KVM Guest:
- virtual console;
- vnc.
Console
To connect using console:
[admin@proxy] /kvm> console ROS
You will see your newly added virtual interface here:
[admin@mr0] > interface print Flags: D - dynamic, X - disabled, R - running, S - slave # NAME TYPE MTU 0 R ether1 ether 1500
To disconnect from the metarouter virtual machine console, hit CTRL + A and then Q to Quit back to your Host console (if you are using minicom, hit CTRL + A twice):
[admin@MikroTik] > [Q - quit connection] [B - send break] [A - send Ctrl-A prefix] [R - autoconfigure rate] Q Welcome back!
VNC
Before connecting with VNC client guest needs some configuration changes.
[admin@proxy] /kvm> print
Flags: X - disabled
0 name="ROS" cpu-count=2 memory=128MiB disk-images=hda:ros1.img kernel="/boot/vmlinuz"
kernel-cmdline="" initrd="/boot/initrd.rgz" vnc-server=0.0.0.0:0 snapshot=no state=running
[admin@proxy] /kvm> shut-down 0
[admin@proxy] /kvm> set 0 vnc-server=10.5.100.99:1
[admin@proxy] /kvm> start 0
[admin@proxy] /kvm> print
Flags: X - disabled
0 name="ROS" cpu-count=2 memory=128MiB disk-images=hda:ros1.img kernel="/boot/vmlinuz"
kernel-cmdline="" initrd="/boot/initrd.rgz" vnc-server=10.5.100.99:1 snapshot=no
state=running
[admin@proxy] /kvm>
VNC servers address in this case is the address on the host reachable from remote locations. Address is followed by screen number.
Now we can try to connect from remote location:
mrz@bumba:/$ vncviewer 10.5.100.99:1
Configuring a virtual network
Right now you saw that the virtual interface is visible in the Host Interfaces menu as tap1 and also in the guest interfaces menu as ether1. You can add an IP address on both interfaces, and set up networking. Creating a bridge between the virtual interface and a physical interface allows traffic to pass.
As an example lets make three virtual routers connected to each other on the same broadcast domain.
File:Kvm-test-bcast.png
Create images and guests:
/kvm make-routeros-image file-name=R1.img file-size=64 make-routeros-image file-name=R2.img file-size=64 make-routeros-image file-name=R3.img file-size=64 add name=R1 disk-image=hda:R1.img add name=R2 disk-image=hda:R2.img add name=R3 disk-image=hda:R3.img
Create a bridge interface which will simulate broadcast domain and add virtual interfaces:
/interface bridge add name=kvm_bridge /kvm interface add virtual-machine=R1 type=dynamic dynamic-bridge=kvm_bridge add virtual-machine=R2 type=dynamic dynamic-bridge=kvm_bridge add virtual-machine=R3 type=dynamic dynamic-bridge=kvm_bridge
Now we can start virtual machines and verify if dynamic interfaces are created:
[admin@proxy] /kvm> start [find]
[admin@proxy] > /interface virtual-ethernet print Flags: D - dynamic, X - disabled, R - running # NAME MTU ARP MAC-ADDRESS 0 D R tap2 1500 enabled 02:20:94:67:D6:D5 1 D R tap3 1500 enabled 02:95:EE:EA:43:FF 2 D R tap4 1500 enabled 02:05:7E:4B:86:F9 [admin@proxy] > /interface bridge port print Flags: X - disabled, I - inactive, D - dynamic # INTERFACE BRIDGE PRIORITY PATH-COST HORIZON 0 D tap2 kvm_bridge 0x80 10 none 1 D tap3 kvm_bridge 0x80 10 none 2 D tap4 kvm_bridge 0x80 10 none [admin@proxy] >
Now we can connect with console to each of guests and set up ip addresses from the same network and verify reachability. R1
[admin@proxy] > /kvm console R1 [Ctrl-A is the prefix key] MikroTik 5.0rc8 MikroTik Login: admin Password: [admin@MikroTik] > /ip address add address=192.168.1.1/24 interface=ether1
R2
<pre> [admin@proxy] > /kvm console R2 [Ctrl-A is the prefix key] MikroTik 5.0rc8 MikroTik Login: admin Password: [admin@MikroTik] > /ip address add address=192.168.1.2/24 interface=ether1
R3
<pre>
[admin@proxy] > /kvm console R1
[Ctrl-A is the prefix key]
MikroTik 5.0rc8
MikroTik Login: admin
Password:
[admin@MikroTik] > /ip address add address=192.168.1.3/24 interface=ether1
[admin@MikroTik] > /ping 192.168.1.1
HOST SIZE TTL TIME STATUS
192.168.1.1 56 64 11ms
192.168.1.1 56 64 2ms
sent=2 received=2 packet-loss=0% min-rtt=2ms avg-rtt=6ms max-rtt=11ms
[admin@MikroTik] > /ping 192.168.1.2
HOST SIZE TTL TIME STATUS
192.168.1.2 56 64 12ms
sent=1 received=1 packet-loss=0% min-rtt=12ms avg-rtt=12ms max-rtt=12ms
Removing KVM guest
KVM guest has two parts in RouterOS - configuration (kvm, virtual-ethernet, /kvm interface) and image file (/file). If image file is removed, but KVM guest is still running, then file will be removed from file menu but still exist until guest is shut-down or disabled, at that moment file will be removed and storage space returned to available storage on the router.
Reference
General
Sub-menu: /kvm
KVM Guest Properties
To add new KVM guest you will have to issue command add under /kvm menu with attributes as follows:
| Property | Desciption |
|---|---|
| comment (text, default: ') | to add simple text description of the KVM guest |
| cpu-count (1 .. 32, default: 1) | available count of processing cores for guest. Allowed values are [1..32] |
| disabled (yes | no, default: no) | to set guest state after creation, values: yes or no |
| disk-images ( list of images used in guest) | list of image assignment to drives for guest OS. If type will be set to cdrom then guest will automatically boot from that, instead of any other drive configured in this field. It can be single drive specified
disk-images=hda:ros.img or it can be comma seperated list: disk-images=hda:system.img,hdb:swap.img |
| initrd (path) | path to initrd file, can be left empty if running RouterOS as guest |
| kernel (path) | path to kernel image file, if using RouterOS image created on host this field can be left empty |
| kernel-cmdline (text) | parameters that are passed to kernel, it is space separated string. |
| memory (integer default:32) | to set up amount of memory that is available to KVM guest |
| name (text) | name of KVM guest that it will be accessible though the system |
| snapshot (yes | no) | will try to run virtual machine with image file in read-only mode. |
| vnc-server-address (IP address ) | address to bind VNC server port that will connect to guest virtual screen. If left empty it will bind to all IP addresses. |
| vnc-server-display (number (0..99) default:0) | will try to run virtual machine with image file in read-only |
| copy-from (number) | use configuration from already existing KVM guest |
Warning: vnc-server attribute has been changed since RouterOS 5.0. in older versions instead of vnc-server-address and vnc-server-display was used combine attribute named vnc-server <IP address>:<display number>
States of KVM guest
This field is read-only and is set by RouterOS. These are possible values that can be set:
- stopped - KVM guest is not running, either successful shut-down or disabled.
- stopping - KVM guest is shutting down
- starting - KVM guest is starting
- running - KVM guest has started successfully and is executing guest operating system
- restarting - KVM guest is reloading its guest operating system
- failed - KVM guest has encountered an error and is not operational.
- image-busy - image file set in configuration is already in use by other KVM guest entry
- no-kernel-or-initrd - initrd or kernel was not found in files set in configuration, mentioned files could not be found or no values in those fields where set
- no-disk-image - either disk image was not found or disk image was not set in configuration.
- kernel-extract-failed - when in guest configuration field kernel is left empty and and KVM cannot extract kernel from image file supplied
- vnc-cant-bind - vnc server for guest cannot bind to setting specified in vnc-server-address and/or vnc-server-display
KVM commands
Sub-menu allows to manage KVM guests on RouterOS host.
| Command | Desciption |
|---|---|
| add | Create new KVM guest entry |
| comment | Set comment for KVM guest entry |
| console | to connect to KVM guest console display |
| continue | resume KVM guest if it was paused |
| disable | change global state of KVM guest. If enabled KVM guest will be started when RouterOS boots. KVM guest cannot change |
| edit | edit selected value of KVM guest entry |
| enable | change KVM guest global state to enable operation of KVM guest. If guest where disabled before - KVM guest is automatically started. |
| export | Print or save an export script that can be used to restore configuration of current sub-menu, KVM guest configuration, image files will not be saved |
| find | Find items by value |
| get | Gets value of item's property |
| make-routeros-image | creates RouterOS image from current installation installed on the router with no configuration. It is advised to create Image file larger than minimal, so you are able to upload new package files and upgrade/update RouterOS installation. Also, all the additional files created in KVM guest will be stored in file image. This image file is not connected to host RouterOS and user is able to run different RouterOS versions on host and guest. This command will create RAW image file containing RouterOS installation.
parameters:
|
| pause | suspend operation of KVM guest |
| Print values of item properties | |
| reboot | issue ACPI shut-down command to KVM guest, if guest does not support ACPI, command have no effect. After KVM guest is shut-downed it will be automatically started by host when shut down is complete. |
| reconfigure-routeros-image | sets up default configuration for RouterOS image.
Parameters:
|
| remove | Remove item |
| set | Change item properties |
| shut-down | issue ACPI shut-down command to KVM guest, if guest does not support ACPI, command have no effect. |
| start | to start KVM guest |
Interface
Sub-menu: /kvm interface
| Property | Desciption |
|---|---|
| comment (text) | description of interface |
| disabled (yes|no, default: no) | state of interface after creation |
| host-mac-address (MAC Address, default: generated) | MAC address of virtual interface that host will use |
| model (virto | e1000 | pcnet, default: virtio) | mode of virtual interface. Available options are:
|
| vm-mac-address (MAC Address, default: generated) | MAC address of virtual interface that guest will use |
| copy-from (number) | use configuration from existing virtual interface |
| dynamic-bridge (interface name, default: none) | if set, dynamic interface will be automatically added as port to bridge interface |
| interface | is set for static interface, to assign it to already created virtual-ethernet interface |
| type (dynamic | static, default: static) | to set if interface is either static or dynamic.
|
| virtual-machine (KVM machine name, must be set) | name of virtual machine this interface will be assigned to |