Manual:Performance Testing with Traffic Generator
Summary
RouterOS Version 6 introduced new tool "traffic generator", which allows to perform performance testing without expensive testing hardware. Traffic is generated from one more router in the network.
This article shows necessary configuration and hardware to perform the same tests published in www.routerboard.com.
RB1100AHx2 Test setup
First step is to choose which ports we will be using for testing.
If we look at the diagram how ports are connected to CPU, fastest combinations are:
- port from switch1 to port form switch chip2,
- ether11 to switch chip,
- ether12/13 to switch chip or to ether11.
To get the maximum out of RB1100AHx2 we will be running 6 streams in total:
- from ether1 to ether6
- from ether1 to ether11
- from ether6 to ether1
- from ether6 to ether11
- from ether11 to ether6
- from ether11 to ether1
In our test environment one RB1100AHx2 will be device under test (DUT) and other RB1100AHx2 will be Traffic generator device.
Connect ether1 to ether1, ether6 to ether6, ether11 to ether11 and proceed with software configuration. Either it will be routing (layer3) testing or bridging (layer2) testing.
Routing Performance Testing
DUT Config
/ip address add address=1.1.1.254/24 interface=ether1 network=1.1.1.0 add address=2.2.2.254/24 interface=ether6 network=2.2.2.0 add address=3.3.3.254/24 interface=ether11 network=3.3.3.0
Traffic Generator Config
/ip address add address=1.1.1.1/24 interface=ether1 network=1.1.1.0 add address=2.2.2.2/24 interface=ether6 network=2.2.2.0 add address=3.3.3.3/24 interface=ether11 network=3.3.3.0 /tool traffic-generator packet-template add name=r12 header-stack=mac,ip,udp ip-gateway=1.1.1.254 ip-dst=2.2.2.2 add name=r13 header-stack=mac,ip,udp ip-gateway=1.1.1.254 ip-dst=3.3.3.3 add name=r21 header-stack=mac,ip,udp ip-gateway=2.2.2.254 ip-dst=1.1.1.1 add name=r23 header-stack=mac,ip,udp ip-gateway=2.2.2.254 ip-dst=3.3.3.3 add name=r32 header-stack=mac,ip,udp ip-gateway=3.3.3.254 ip-dst=2.2.2.2 add name=r31 header-stack=mac,ip,udp ip-gateway=3.3.3.254 ip-dst=1.1.1.1
Note: To force MAC address re-discovery (on device/configuration change, just apply emply "set" command to necessary packet-templates)
Running Tests
/tool traffic-generator quick tx-template=r12,r13,r21,r23,r31,r32 packet-size=60 mbps=300
We are specifying 60byte packet in traffic generator to get 64 byte packet on ethernet.
[admin@TrafficGen] > /tool traffic-gen quick tx-template=r12,r13,r21,r23,r31,r32 packet-size=60 mbps=500 SEQ ID TX-PACKET TX-RATE RX-PACKET RX-RATE RX-OOO LOST-PACKET LOST-RATE LAT-MIN ... TOT 0 666 013 110.1Mbps 665 401 106.4Mbps 612 3.6Mbps 21.9us TOT 1 797 870 131.9Mbps 780 769 124.9Mbps 17 101 6.9Mbps 11.7us TOT 2 693 307 114.6Mbps 691 959 110.7Mbps 1 348 3.9Mbps 22.3us TOT 3 805 247 133.1Mbps 781 710 125.0Mbps 23 537 8.0Mbps 12.4us TOT 4 839 077 134.2Mbps 403 396 64.5Mbps 435 681 69.7Mbps 29.9us TOT 5 839 043 134.2Mbps 413 945 66.2Mbps 425 098 68.0Mbps 14.9us
TOT TOT 4 640 557 758.2Mbps 3 737 180 597.9Mbps 903 377 160.3Mbps 11.7us
You can also check in the DUT if forwarding is actually happening:
[admin@DUT] > /interface monitor-traffic aggregate,ether1,ether6,ether11
name: ether1 ether6 ether11
rx-packets-per-second: 1 235 620 481 094 487 045 267 469
rx-drops-per-second: 0 0 0 0
rx-errors-per-second: 0 0 0 0
rx-bits-per-second: 593.0Mbps 230.9Mbps 233.7Mbps 128.3Mbps
tx-packets-per-second: 1 233 862 360 750 360 402 512 692
tx-drops-per-second: 0 0 0 0
tx-errors-per-second: 0 0 0 0
tx-bits-per-second: 603.9Mbps 178.9Mbps 178.7Mbps 246.0Mbps
After running the test you can see that total throughput of 64byte packets are 598Mbps which is almost twice as fast than shown in routerboard.com results.
This is because of by default enabled fast-path mode.
Lets enable connection tracking on DUT:
/ip firewall connection tracking set enabled=yes
And run the test again. As you can see now it is almost the same value as advertised.
[admin@TrafficGen] > /tool traffic-generator quick tx-template=r12,r13,r21,r23,r31,r32 packet-size=60 mbps=5 00 SEQ ID TX-PACKET TX-RATE RX-PACKET RX-RATE RX-OOO LOST-PACKET LOST-RATE LAT-MIN ... TOT 0 1 970 004 162.8Mbps 767 734 61.4Mbps 1 202 270 101.4Mbps 316us TOT 1 2 307 840 190.7Mbps 512 347 40.9Mbps 1 795 493 149.7Mbps 37.5us TOT 2 1 980 927 163.7Mbps 763 490 61.0Mbps 1 217 437 102.6Mbps 2.94ms TOT 3 2 310 745 191.0Mbps 509 504 40.7Mbps 1 801 241 150.2Mbps 19.1us TOT 4 2 360 869 188.8Mbps 247 539 19.8Mbps 2 113 330 169.0Mbps 2.74ms TOT 5 2 360 743 188.8Mbps 261 714 20.9Mbps 2 099 029 167.9Mbps 963us
TOT TOT 13 291 128 1086.1... 3 062 328 244.9Mbps 10 228 800 125.3Mbps 19.1us
We can now add more firewall rules, queues and any other configuration and see how much router can actually handle.
Lets add some firewall rules
We will take the customer protection rules from the manual
Start by adding default rules that should present on any firewall:
/ip firewall filter add chain=forward protocol=tcp connection-state=invalid \ action=drop comment="drop invalid connections" add chain=forward connection-state=established action=accept \ comment="allow already established connections" add chain=forward connection-state=related action=accept \ comment="allow related connections"
We get approximately 40Mbps less
TOT TOT 13 926 377 1138.0... 2 586 802 206.9Mbps 11 339 575 215.3Mbps 41.6us
Now add more rules from the manual to see how count of firewall rules influence performance of the board
/ip firewall filter add chain=forward protocol=icmp action=jump jump-target=icmp add chain=icmp protocol=icmp icmp-options=0:0 action=accept \ comment="echo reply" add chain=icmp protocol=icmp icmp-options=3:0 action=accept \ comment="net unreachable" add chain=icmp protocol=icmp icmp-options=3:1 action=accept \ comment="host unreachable" add chain=icmp protocol=icmp icmp-options=3:4 action=accept \ comment="host unreachable fragmentation required" add chain=icmp protocol=icmp icmp-options=4:0 action=accept \ comment="allow source quench" add chain=icmp protocol=icmp icmp-options=8:0 action=accept \ comment="allow echo request" add chain=icmp protocol=icmp icmp-options=11:0 action=accept \ comment="allow time exceed" add chain=icmp protocol=icmp icmp-options=12:0 action=accept \ comment="allow parameter bad" add chain=icmp action=drop comment="deny all other types"
TOT TOT 16 309 055 1142.4... 2 978 839 204.2Mbps 13 330 216 324.5Mbps 53.3us
There are almost no performance changes.
You can add further any amount of rules and see that there are minimum influence on performance of the router.
Perform the same test with different packet sizes:
/tool traffic-generator quick tx-template=r12,r13,r21,r23,r31,r32 packet-size=508 mbps=500 /tool traffic-generator quick tx-template=r12,r13,r21,r23,r31,r32 packet-size=1514 mbps=500
Note: mind that speed in quick mode is specified per stream, so if you have two streams per port, you need to send 1/2 of traffic per stream
Bridging Performance Testing
DUT Config
/interface bridge add /interface bridge port add bridge=bridge1 interface=ether1 add bridge=bridge1 interface=ether6 add bridge=bridge1 interface=ether11
Traffic Generator Config
/ip address add address=1.1.1.1/24 interface=ether1 network=1.1.1.0 add address=2.2.2.2/24 interface=ether6 network=2.2.2.0 add address=3.3.3.3/24 interface=ether11 network=3.3.3.0 /tool traffic-generator packet-template add header-stack=mac,ip,udp ip-src=1.1.1.1/32 ip-dst=2.2.2.2/32 name=b12 add header-stack=mac,ip,udp ip-src=1.1.1.1/32 ip-dst=3.3.3.3/32 name=b13 add header-stack=mac,ip,udp ip-src=2.2.2.2/32 ip-dst=1.1.1.1/32 name=b21 add header-stack=mac,ip,udp ip-src=2.2.2.2/32 ip-dst=3.3.3.3/32 name=b23 add header-stack=mac,ip,udp ip-src=3.3.3.3/32 ip-dst=1.1.1.1/32 name=b31 add header-stack=mac,ip,udp ip-src=3.3.3.3/32 ip-dst=2.2.2.2/32 name=b32
Running Tests
/tool traffic-generator quick tx-template=b12,b13,b21,b23,b31,b32 packet-size=60 mbps=200 /tool traffic-generator quick tx-template=b12,b13,b21,b23,b31,b32 packet-size=508 mbps=500 /tool traffic-generator quick tx-template=b12,b13,b21,b23,b31,b32 packet-size=1514 mbps=500