Manual:Performance Testing with Traffic Generator: Difference between revisions

Latest revision as of 13:21, 1 June 2020

Summary

RouterOS version 6 introduces a new tool - "traffic generator", which allows performing performance testing without expensive testing hardware. Traffic is generated from one more router in the network.

This article shows the necessary configuration and hardware to replicate the tests published in routerboard.com.

RB1100AHx2 Test setup

The first step is to choose which ports we will be using for testing.

If we look at the diagram how ports are connected to CPU, the fastest combinations are:

port from switch1 to port form switch chip2,
ether11 to switch chip,
ether12/13 to switch chip or to ether11.

To get the maximum out of RB1100AHx2 we will be running 6 streams in total:

from ether1 to ether6
from ether1 to ether11
from ether6 to ether1
from ether6 to ether11
from ether11 to ether6
from ether11 to ether1

In our test environment, one RB1100AHx2 will be a device under test (DUT) and other RB1100AHx2 will be a Traffic generator device.

Connecting the routers

Note: RouterOS v6 should be used on both test routers.

Connect cables like this: ether1 to ether1, ether6 to ether6, ether11 to ether11

Note: Ether12 will be added where test reaches wire speed.

Now proceed with software configuration. Either it will be routing (layer3) testing or bridging (layer2) testing.

Routing Performance Testing

DUT Config

/ip address
add address=192.168.86.254/24 interface=ether1 network=192.168.86.0
add address=192.168.87.254/24 interface=ether6 network=192.168.87.0
add address=192.168.88.254/24 interface=ether11 network=192.168.88.0

Traffic Generator Config

/ip address
add address=192.168.86.1/24 interface=ether1 network=192.168.86.0
add address=192.168.87.1/24 interface=ether6 network=192.168.87.0
add address=192.168.88.1/24 interface=ether11 network=192.168.88.0

/tool traffic-generator packet-template
add name=r12 header-stack=mac,ip,udp ip-gateway=192.168.86.254 ip-dst=192.168.87.1
add name=r13 header-stack=mac,ip,udp ip-gateway=192.168.86.254 ip-dst=192.168.88.1
add name=r21 header-stack=mac,ip,udp ip-gateway=192.168.87.254 ip-dst=192.168.86.1
add name=r23 header-stack=mac,ip,udp ip-gateway=192.168.87.254 ip-dst=192.168.88.1
add name=r32 header-stack=mac,ip,udp ip-gateway=192.168.88.254 ip-dst=192.168.87.1
add name=r31 header-stack=mac,ip,udp ip-gateway=192.168.88.254 ip-dst=192.168.86.1

Note: To force MAC address re-discovery (on device/configuration change, just apply emply "set" command to necessary packet-templates)

Running Tests

/tool traffic-generator
quick tx-template=r12,r13,r21,r23,r31,r32 packet-size=60 mbps=300

Note: We are specifying 60 byte packet in traffic generator to get a 64 byte packet on ethernet.

[admin@TrafficGen] > /tool traffic-gen quick tx-template=r12,r13,r21,r23,r31,r32 packet-size=60 
mbps=120

24     0         185 422  91.9Mbps       185 190  88.8Mbps                     232   3.0Mbps 16us   
24     1         213 397 105.8Mbps       212 747 102.1Mbps                     650   3.7Mbps 10.6us 
24     2         186 245  92.3Mbps       186 185  89.3Mbps                      60   3.0Mbps 16.4us 
24     3         213 685 105.9Mbps       212 961 102.2Mbps                     724   3.7Mbps 10.8us 
24     4         249 142 119.5Mbps       180 400  86.5Mbps                  68 742  32.9Mbps 13.2us 
24     5         249 141 119.5Mbps       193 158  92.7Mbps                  55 983  26.8Mbps 11.1us 
24     TOT     1 297 032 635.3Mbps     1 170 641 561.9Mbps                 126 391  73.4Mbps 10.6us

You can also check in the DUT if forwarding is actually happening:

[admin@DUT] > /interface monitor-traffic aggregate,ether1,ether6,ether11
                     name:               ether1    ether6   ether11
    rx-packets-per-second:  1 235 620   481 094   487 045   267 469
      rx-drops-per-second:          0         0         0         0
     rx-errors-per-second:          0         0         0         0
       rx-bits-per-second:  593.0Mbps 230.9Mbps 233.7Mbps 128.3Mbps
    tx-packets-per-second:  1 233 862   360 750   360 402   512 692
      tx-drops-per-second:          0         0         0         0
     tx-errors-per-second:          0         0         0         0
       tx-bits-per-second:  603.9Mbps 178.9Mbps 178.7Mbps 246.0Mbps

After running the test you can see that the total throughput of 64byte packets is 1'170'641pps which is a lot faster than shown in routerboard.com results.

This is because by default fast-path mode is enabled.

Lets enable connection tracking on DUT:

/ip firewall connection tracking set enabled=yes

And run the test again. As you can see now it is close to the advertised PPS rate.

46     0         249 793 123.8Mbps       127 410  61.1Mbps                 122 383  62.7Mbps 3.22ms 
46     1         249 791 123.8Mbps        87 232  41.8Mbps                 162 559  82.0Mbps 5.2ms  
46     2         249 792 123.8Mbps       127 424  61.1Mbps                 122 368  62.7Mbps 3.15ms 
46     3         249 792 123.8Mbps        87 219  41.8Mbps                 162 573  82.0Mbps 5.18ms 
46     4         249 792 119.9Mbps        40 492  19.4Mbps                 209 300 100.4Mbps 5.54ms 
46     5         249 791 119.8Mbps        46 736  22.4Mbps                 203 055  97.4Mbps 5.41ms 
46     TOT     1 498 751 735.3Mbps       516 513 247.9Mbps                 982 238 487.4Mbps 3.15ms

We can now add more firewall rules, queues, and any other configuration and see how many routers can actually handle.

Lets add some firewall rules

We will take the customer protection rules from the manual

Start by adding default rules that should present on any firewall:

/ip firewall filter
add chain=forward protocol=tcp connection-state=invalid \
	action=drop comment="drop invalid connections"  
add chain=forward connection-state=established action=accept \
	comment="allow already established connections"  
add chain=forward connection-state=related action=accept \
	comment="allow related connections"

We get approximately 18% less packets

53     TOT     1 492 520 732.3Mbps       435 546 209.0Mbps               1 056 974 523.2Mbps 3.08ms

Now add more rules from the manual to see how the count of firewall rules affects the performance of the board

/ip firewall filter
add chain=forward protocol=icmp action=jump jump-target=icmp 

add chain=icmp protocol=icmp icmp-options=0:0 action=accept \
 	comment="echo reply"  
add chain=icmp protocol=icmp icmp-options=3:0 action=accept \
 	comment="net unreachable"  
add chain=icmp protocol=icmp icmp-options=3:1 action=accept \
 	comment="host unreachable"
add chain=icmp protocol=icmp icmp-options=3:4 action=accept \
 	comment="host unreachable fragmentation required"  
add chain=icmp protocol=icmp icmp-options=4:0 action=accept \
 	comment="allow source quench"  
add chain=icmp protocol=icmp icmp-options=8:0 action=accept \
 	comment="allow echo request"  
add chain=icmp protocol=icmp icmp-options=11:0 action=accept \
 	comment="allow time exceed"  
add chain=icmp protocol=icmp icmp-options=12:0 action=accept \
 	comment="allow parameter bad"  
add chain=icmp action=drop comment="deny all other types"

33     TOT     1 500 908 736.4Mbps       424 197 203.6Mbps               1 076 711 532.8Mbps 4.07ms

There are almost no performance changes. You can add any amount of rules and see that there is only a small influence on the performance of the router.

Perform the same test with different packet sizes:

/tool traffic-generator
quick tx-template=r12,r13,r21,r23,r31,r32 packet-size=508 mbps=500
/tool traffic-generator
quick tx-template=r12,r13,r21,r23,r31,r32 packet-size=1514 mbps=500

If we run the test with 1518 packet size then max throughput will be only 2.9Gbps This is because wire speed of all interfaces are reached.

We will need to add one more port to our test and add streams.

Connect ether12 to ether12 and proceed with configuration

On DUT:

/ip address 
add address=192.168.89.254/24 interface=ether12

On TrafficGen

/ip address
add address=192.168.89.254/24 interface=ether12

/tool traffic-generator packet-template
add header-stack=mac,ip,udp ip-dst=192.168.89.254/32 ip-gateway=192.168.86.254 ip-src=192.168.86.1/32 name=\
    r14
add header-stack=mac,ip,udp ip-dst=192.168.89.254/32 ip-gateway=192.168.87.254 ip-src=192.168.87.1/32 name=\
    r24
add header-stack=mac,ip,udp ip-dst=192.168.89.254/32 ip-gateway=192.168.88.254 ip-src=192.168.88.1/32 name=\
    r34
add header-stack=mac,ip,udp ip-dst=192.168.86.1/32 ip-gateway=192.168.89.254 ip-src=192.168.89.254/32 name=\
    r41
add header-stack=mac,ip,udp ip-dst=192.168.87.1/32 ip-gateway=192.168.89.254 ip-src=192.168.89.254/32 name=\
    r42
add header-stack=mac,ip,udp ip-dst=192.168.88.1/32 ip-gateway=192.168.89.254 ip-src=192.168.89.254/32 name=\
    r43

And now run the test:

/tool traffic-generator quick tx-template=r12,r13,r14,r21,r23,r24,r31,r32,r34,r41,r42,r43 \
 packet-size=1514 mbps=350

30     6          23 472 284.2Mbps        23 328 282.5Mbps                     144 1744.1... 3.22ms 
30     7          28 890 349.9Mbps        28 741 348.1Mbps                     149 1804.6... 1.74ms 
30     8          28 889 349.9Mbps        26 870 325.4Mbps                   2 019  24.4Mbps 984us  
30     9          23 455 284.0Mbps        23 083 279.5Mbps                     372   4.5Mbps 866us  
30     10         28 876 349.7Mbps        28 709 347.7Mbps                     167   2.0Mbps 922us  
30     11         28 875 349.7Mbps        27 277 330.3Mbps                   1 598  19.3Mbps 3.33ms 
30     TOT       323 389   3.9Gbps       311 743   3.7Gbps                  11 646 143.6Mbps 341us

As you can see we get 3.7Gbps.

And with all firewalls enabled from previous tests we get 2.8Gbps which is approximately 30% slower:

18     TOT       275 405   3.3Gbps       238 143   2.8Gbps                 37 262 453.9Mbps 1.57ms

Note: mind that speed in quick mode is specified per stream, so if you have two streams per port, you need to send 1/2 of traffic per stream

Bridging Performance Testing

DUT Config

/interface bridge add
/interface bridge port
add bridge=bridge1 interface=ether1
add bridge=bridge1 interface=ether6
add bridge=bridge1 interface=ether11

Traffic Generator Config

/ip address
add address=192.168.86.1/24 interface=ether1 network=192.168.86.0
add address=192.168.87.1/24 interface=ether6 network=192.168.87.0
add address=192.168.88.1/24 interface=ether11 network=192.168.88.0

/tool traffic-generator packet-template
add header-stack=mac,ip,udp ip-src=192.168.86.1/32 ip-dst=192.168.87.1/32 name=b12
add header-stack=mac,ip,udp ip-src=192.168.86.1/32 ip-dst=192.168.88.1/32 name=b13
add header-stack=mac,ip,udp ip-src=192.168.87.1/32 ip-dst=192.168.86.1/32 name=b21
add header-stack=mac,ip,udp ip-src=192.168.87.1/32 ip-dst=192.168.88.1/32 name=b23
add header-stack=mac,ip,udp ip-src=192.168.88.1/32 ip-dst=192.168.86.1/32 name=b31
add header-stack=mac,ip,udp ip-src=192.168.88.1/32 ip-dst=192.168.87.1/32 name=b32

Running Tests

/tool traffic-generator
quick tx-template=b12,b13,b21,b23,b31,b32 packet-size=60 mbps=200
/tool traffic-generator
quick tx-template=b12,b13,b21,b23,b31,b32 packet-size=508 mbps=500
/tool traffic-generator
quick tx-template=b12,b13,b21,b23,b31,b32 packet-size=1514 mbps=500

With small packets, we get approximately 1.4 mil packets per second

187    0         195 659  97.0Mbps       195 640  93.9Mbps                      19   3.1Mbps 22us   
187    1         236 906 117.5Mbps       221 901 106.5Mbps                  15 005  10.9Mbps 18.7us 
187    2         202 678 100.5Mbps       202 678  97.2Mbps                       0   3.2Mbps 18.7us 
187    3         238 750 118.4Mbps       231 348 111.0Mbps                   7 402   7.3Mbps 12.1us 
187    4         263 906 126.6Mbps       256 146 122.9Mbps                   7 760   3.7Mbps 23.9us 
187    5         263 906 126.6Mbps       256 030 122.8Mbps                   7 876   3.7Mbps 14.3us 
187    TOT     1 401 805 686.8Mbps     1 363 743 654.5Mbps                  38 062  32.2Mbps 12.1us

With 1518 byte packets we will get wire-speed maximum

11     TOT       243 587   2.9Gbps       241 695   2.9Gbps                   1 892  25.5Mbps 1.04ms

So we will need to use ether12 and add a few more streams just like in the routing test.

CCR1036-8G-2S+ Test setup

CCR1036 series routers have very powerful CPU, So the test will use all ports. As shown in the diagram below

The test will be separated in units, each unit will hold a pair of interfaces. Here is the list of units and what medium is used:

sfp-sfpplus1 will pass data to sfp-sfpplus2 (direct attach copper 10GBit)
ether1 to ether2 (cat5e)
ether3 to ether4 (cat5e)
ether5 to ether6 (cat5e)
ether7 to ether8 (cat5e)

For testing we will require 2 other CCR1036 as passing data through is not as intensive as generating traffic and gathering statistical data.

Note: For a test with large packets only one additional CCR1036 is required for use with the traffic-generator tool. All the configuration examples will not interfere with each other and can be imported on one testing router

Connect TR1 (Trafic-generator Router 1) to DUT:

SFP/SFP+1 to SFP/SFP+1 Direct attach 10GBit cable (10Gbit optical SFP interfaces cable used instead)
SFP/SFP+2 to SFP/SFP+2 Direct attach 10GBit cable (10Gbit optical SFP interfaces cable used instead)

Connect TR2 to DUT:

ether1 to ether1
ether2 to ether2
ether3 to ether3
ether4 to ether4
ether5 to ether5
ether6 to ether6
ether7 to ether7
ether8 to ether8

Defaults, routing

The default configuration of DUT is with enabled fast-path

DUT configuration

 /ip address
 add address=10.0.100.1/24 interface=sfp-sfpplus1 network=10.0.100.0
 add address=10.0.101.1/24 interface=sfp-sfpplus2 network=10.0.101.0
 add address=10.0.110.1/24 interface=ether1 network=10.0.110.0
 add address=10.0.111.1/24 interface=ether2 network=10.0.111.0
 add address=10.0.112.1/24 interface=ether3 network=10.0.112.0
 add address=10.0.113.1/24 interface=ether4 network=10.0.113.0
 add address=10.0.114.1/24 interface=ether5 network=10.0.114.0
 add address=10.0.115.1/24 interface=ether6 network=10.0.115.0
 add address=10.0.116.1/24 interface=ether7 network=10.0.116.0
 add address=10.0.117.1/24 interface=ether8 network=10.0.117.0

TR1 configuration (10Gbit interfaces)

 /ip address
 add address=10.0.100.2/24 interface=sfp-sfpplus1 network=10.0.100.0
 add address=10.0.101.2/24 interface=sfp-sfpplus2 network=10.0.101.0

 /tool traffic-generator packet-template
 add header-stack=mac,ip,udp interface=sfp-sfpplus1 ip-dst=10.0.101.2 ip-gateway=10.0.100.1 name=pt10
 add header-stack=mac,ip,udp interface=sfp-sfpplus2 ip-dst=10.0.100.2 ip-gateway=10.0.101.1 name=pt11

 /tool traffic-generator stream
 add id=0 mbps=6000 name=str10 packet-size=60 tx-template=pt10
 add id=1 mbps=6000 name=str11 packet-size=60 tx-template=pt11

TR2 configuration (1Gbit interfaces)

 /ip address
 add address=10.0.110.2/24 interface=ether1 network=10.0.110.0
 add address=10.0.111.2/24 interface=ether2 network=10.0.111.0
 add address=10.0.112.2/24 interface=ether3 network=10.0.112.0
 add address=10.0.113.2/24 interface=ether4 network=10.0.113.0
 add address=10.0.114.2/24 interface=ether5 network=10.0.114.0
 add address=10.0.115.2/24 interface=ether6 network=10.0.115.0
 add address=10.0.116.2/24 interface=ether7 network=10.0.116.0
 add address=10.0.117.2/24 interface=ether8 network=10.0.117.0

 /tool traffic-generator packet-template
 add header-stack=mac,ip,udp ip-dst=10.0.111.2 ip-gateway=10.0.110.1 name=pt1
 add header-stack=mac,ip,udp ip-dst=10.0.110.2 ip-gateway=10.0.111.1 name=pt2
 add header-stack=mac,ip,udp ip-dst=10.0.113.2 ip-gateway=10.0.112.1 name=pt3
 add header-stack=mac,ip,udp ip-dst=10.0.112.2 ip-gateway=10.0.113.1 name=pt4
 add header-stack=mac,ip,udp ip-dst=10.0.115.2 ip-gateway=10.0.114.1 name=pt5
 add header-stack=mac,ip,udp ip-dst=10.0.114.2 ip-gateway=10.0.115.1 name=pt6
 add header-stack=mac,ip,udp ip-dst=10.0.117.2 ip-gateway=10.0.116.1 name=pt7
 add header-stack=mac,ip,udp ip-dst=10.0.116.2 ip-gateway=10.0.117.1 name=pt8

 /tool traffic-generator stream
 add id=0 mbps=700 name=str0 packet-size=60 tx-template=pt0
 add id=1 mbps=700 name=str1 packet-size=60 tx-template=pt1
 add id=2 mbps=700 name=str3 packet-size=60 tx-template=pt2
 add id=3 mbps=700 name=str4 packet-size=60 tx-template=pt3
 add id=4 mbps=700 name=str5 packet-size=60 tx-template=pt4
 add id=5 mbps=700 name=str6 packet-size=60 tx-template=pt5
 add id=6 mbps=700 name=str7 packet-size=60 tx-template=pt6
 add id=7 mbps=700 name=str8 packet-size=60 tx-template=pt7

Running test

To run tests use ON TR1

 /tool traffic-generator quick packet-size=60 mbps=6000

On TR2

 /tool traffic-generator quick packet-size=60 mbps=700

Profile for medium-sized packets:

ON TR1

 /tool traffic-generator quick packet-size=508 mbps=10000

On TR2

 /tool traffic-generator quick packet-size=508 mbps=1000

Profile for large packets: ON TR1

 /tool traffic-generator quick packet-size=1514 mbps=10000

On TR2

 /tool traffic-generator quick packet-size=1514 mbps=1000

See More

Manual:Performance Testing with Traffic Generator: Difference between revisions

Latest revision as of 13:21, 1 June 2020

Contents

Summary

RB1100AHx2 Test setup

Connecting the routers

Routing Performance Testing

DUT Config

Traffic Generator Config

Running Tests

Bridging Performance Testing

DUT Config

Traffic Generator Config

Running Tests

CCR1036-8G-2S+ Test setup

Defaults, routing

DUT configuration

TR1 configuration (10Gbit interfaces)

TR2 configuration (1Gbit interfaces)

Running test

See More

Navigation menu

@@ Line 3: / Line 3: @@
 ==Summary==
-RouterOS Version 6 introduced new tool "traffic generator", which allows to perform performance testing without expensive testing hardware. Traffic is generated from one more router in the network.
+RouterOS version 6 introduces a new tool - "traffic generator", which allows performing performance testing without expensive testing hardware. Traffic is generated from one more router in the network.
-This article shows necessary configuration and hardware to perform the same tests published in [http://www.routerboard.com www.routerboard.com].
+This article shows the necessary configuration and hardware to replicate the tests published in [http://routerboard.com/RB1100AHx2#tests routerboard.com].
 ==RB1100AHx2 Test setup==
-First step is to choose which ports we will be using for testing.
+The first step is to choose which ports we will be using for testing.
 [[File:RB1100AHx2-diagram-streams.png | center]]
-If we look at the diagram how ports are connected to CPU, fastest combinations are:
+If we look at the diagram how ports are connected to CPU, the fastest combinations are:
 * port from switch1 to port form switch chip2,
 * ether11 to switch chip,
@@ Line 28: / Line 28: @@
-In our test environment one RB1100AHx2 will be device under test (DUT) and other RB1100AHx2 will be Traffic generator device.
+In our test environment, one RB1100AHx2 will be a device under test (DUT) and other RB1100AHx2 will be a Traffic generator device.
-Connect ether1 to ether1, ether6 to ether6, ether11 to ether11 and proceed with software configuration.
+=== Connecting the routers ===
-Either it will be routing (layer3) testing or bridging (layer2) testing.
+{{Note| RouterOS v6 should be used on both test routers.}}
+'''Connect cables like this''': ether1 to ether1, ether6 to ether6, ether11 to ether11
+[[File:1100_outline-wires.png | center |700px]]
+{{Note | Ether12 will be added where test reaches wire speed.}}
+Now proceed with software configuration. Either it will be routing (layer3) testing or bridging (layer2) testing.
 === Routing Performance Testing===
@@ Line 41: / Line 49: @@
 <pre>
 /ip address
-add address=1.1.1.254/24 interface=ether1 network=1.1.1.0
+add address=192.168.86.254/24 interface=ether1 network=192.168.86.0
-add address=2.2.2.254/24 interface=ether6 network=2.2.2.0
+add address=192.168.87.254/24 interface=ether6 network=192.168.87.0
-add address=3.3.3.254/24 interface=ether11 network=3.3.3.0
+add address=192.168.88.254/24 interface=ether11 network=192.168.88.0
 </pre>
@@ Line 51: / Line 59: @@
 <pre>
 /ip address
-add address=1.1.1.1/24 interface=ether1 network=1.1.1.0
+add address=192.168.86.1/24 interface=ether1 network=192.168.86.0
-add address=2.2.2.2/24 interface=ether6 network=2.2.2.0
+add address=192.168.87.1/24 interface=ether6 network=192.168.87.0
-add address=3.3.3.3/24 interface=ether11 network=3.3.3.0
+add address=192.168.88.1/24 interface=ether11 network=192.168.88.0
 /tool traffic-generator packet-template
-add name=r12 header-stack=mac,ip,udp ip-gateway=1.1.1.254 ip-dst=2.2.2.2
+add name=r12 header-stack=mac,ip,udp ip-gateway=192.168.86.254 ip-dst=192.168.87.1
-add name=r13 header-stack=mac,ip,udp ip-gateway=1.1.1.254 ip-dst=3.3.3.3
+add name=r13 header-stack=mac,ip,udp ip-gateway=192.168.86.254 ip-dst=192.168.88.1
-add name=r21 header-stack=mac,ip,udp ip-gateway=2.2.2.254 ip-dst=1.1.1.1
+add name=r21 header-stack=mac,ip,udp ip-gateway=192.168.87.254 ip-dst=192.168.86.1
-add name=r23 header-stack=mac,ip,udp ip-gateway=2.2.2.254 ip-dst=3.3.3.3
+add name=r23 header-stack=mac,ip,udp ip-gateway=192.168.87.254 ip-dst=192.168.88.1
-add name=r32 header-stack=mac,ip,udp ip-gateway=3.3.3.254 ip-dst=2.2.2.2
+add name=r32 header-stack=mac,ip,udp ip-gateway=192.168.88.254 ip-dst=192.168.87.1
-add name=r31 header-stack=mac,ip,udp ip-gateway=3.3.3.254 ip-dst=1.1.1.1
+add name=r31 header-stack=mac,ip,udp ip-gateway=192.168.88.254 ip-dst=192.168.86.1
 </pre>
@@ Line 74: / Line 82: @@
 </pre>
-We are specifying 60byte packet in traffic generator to get 64 byte packet on ethernet.
+{{Note | We are specifying 60 byte packet in traffic generator to get a 64 byte packet on ethernet.}}
+ [admin@TrafficGen] > /tool traffic-gen quick tx-template=r12,r13,r21,r23,r31,r32 packet-size=60
+ mbps=120
+     0         185 422  91.9Mbps       185 190  88.8Mbps                     232   3.0Mbps 16us
+     1         213 397 105.8Mbps       212 747 102.1Mbps                     650   3.7Mbps 10.6us
+     2         186 245  92.3Mbps       186 185  89.3Mbps                      60   3.0Mbps 16.4us
+     3         213 685 105.9Mbps       212 961 102.2Mbps                     724   3.7Mbps 10.8us
+     4         249 142 119.5Mbps       180 400  86.5Mbps                  68 742  32.9Mbps 13.2us
+     5         249 141 119.5Mbps       193 158  92.7Mbps                  55 983  26.8Mbps 11.1us
+     TOT     1 297 032 635.3Mbps     '''1 170 641''' 561.9Mbps                 126 391  73.4Mbps 10.6us
+You can also check in the '''DUT''' if forwarding is actually happening:
 <pre>
-[admin@MikroTik] > /tool traffic-gen quick tx-template=r12,r13,r21,r23,r31,r32 packet-size=60
+[admin@DUT] > /interface monitor-traffic aggregate,ether1,ether6,ether11
-mbps=500
+                     name:               ether1    ether6   ether11
+    rx-packets-per-second:  1 235 620   481 094   487 045   267 469
+      rx-drops-per-second:          0         0         0         0
+     rx-errors-per-second:          0         0         0         0
+       rx-bits-per-second:  593.0Mbps 230.9Mbps 233.7Mbps 128.3Mbps
+    tx-packets-per-second:  1 233 862   360 750   360 402   512 692
+      tx-drops-per-second:          0         0         0         0
+     tx-errors-per-second:          0         0         0         0
+       tx-bits-per-second:  603.9Mbps 178.9Mbps 178.7Mbps 246.0Mbps
+</pre>
-SEQ    ID      TX-PACKET   TX-RATE     RX-PACKET   RX-RATE   RX-OOO   LOST-PACKET LOST-RATE LAT-MIN
-      1         266 517 132.1Mbps       261 599 125.5Mbps                  4 918   6.6Mbps 11.8us
-      2         233 026 115.5Mbps       232 793 111.7Mbps                    233   3.8Mbps 24.2us
-      3         269 786 133.8Mbps       262 371 125.9Mbps                  7 415   7.8Mbps 14.9us
-      4         281 058 134.9Mbps       133 869  64.2Mbps                147 189  70.6Mbps 51us
-      5         281 058 134.9Mbps       137 262  65.8Mbps                143 796  69.0Mbps 45.1us
-      TOT     1 553 859 761.7Mbps     1 250 263 600.1Mbps                303 596 161.5Mbps 11.8us
-      0         220 565 109.4Mbps       220 258 105.7Mbps                    307   3.6Mbps 21.9us
-      1         263 493 130.6Mbps       257 553 123.6Mbps                  5 940   7.0Mbps 14.8us
-      2         230 682 114.4Mbps       230 384 110.5Mbps                    298   3.8Mbps 22.6us
-      3         266 917 132.3Mbps       258 175 123.9Mbps                  8 742   8.4Mbps 24.1us
-      4         277 825 133.3Mbps       134 469  64.5Mbps                143 356  68.8Mbps 43.2us
-      5         277 825 133.3Mbps       138 043  66.2Mbps                139 782  67.0Mbps 35.7us
-      TOT     1 537 307 753.6Mbps     1 238 882 594.6Mbps                298 425 158.9Mbps 14.8us
-TOT    0         666 013 110.1Mbps       665 401 106.4Mbps                    612   3.6Mbps 21.9us
-TOT    1         797 870 131.9Mbps       780 769 124.9Mbps                 17 101   6.9Mbps 11.7us
-TOT    2         693 307 114.6Mbps       691 959 110.7Mbps                  1 348   3.9Mbps 22.3us
-TOT    3         805 247 133.1Mbps       781 710 125.0Mbps                 23 537   8.0Mbps 12.4us
-TOT    4         839 077 134.2Mbps       403 396  64.5Mbps                435 681  69.7Mbps 29.9us
-TOT    5         839 043 134.2Mbps       413 945  66.2Mbps                425 098  68.0Mbps 14.9us
-TOT    TOT     4 640 557 758.2Mbps     3 737 180 597.9Mbps                903 377 160.3Mbps 11.7us
-</pre>
-After running the test you can see that total throughput of 64byte packets are 598Mbps which is almost twice as fast than shown in [http://routerboard.com/RB1100AHx2#tests routerboard.com] results.
-This is because of by default enabled [[M:Fast_Path | fast-path]] mode.
-Lets enable connection tracking on DUT:
+After running the test you can see that the total throughput of 64byte packets is  '''1'170'641'''pps which is a lot faster than shown in [http://routerboard.com/RB1100AHx2#tests routerboard.com] results.
+This is because by default [[M:Fast_Path | fast-path]] mode is enabled.
+'''Lets enable connection tracking on DUT''':
 <pre>
 /ip firewall connection tracking set enabled=yes
 </pre>
-And run the test again. As you can see now it is almost the same value as advertised.
+And run the test again. As you can see now it is close to the advertised PPS rate.
-<pre>
-[admin@MikroTik] > /tool traffic-generator quick tx-template=r12,r13,r21,r23,r31,r32 packet-size=60 mbps=5
+    0         249 793 123.8Mbps       127 410  61.1Mbps                 122 383  62.7Mbps 3.22ms
+     1         249 791 123.8Mbps        87 232  41.8Mbps                 162 559  82.0Mbps 5.2ms
-SEQ    ID      TX-PACKET   TX-RATE     RX-PACKET   RX-RATE        RX-OOO   LOST-PACKET LOST-RATE LAT-MIN
+     2         249 792 123.8Mbps       127 424  61.1Mbps                 122 368  62.7Mbps 3.15ms
-...
+     3         249 792 123.8Mbps        87 219  41.8Mbps                 162 573  82.0Mbps 5.18ms
-TOT    0       1 970 004 162.8Mbps       767 734  61.4Mbps              1 202 270 101.4Mbps 316us
+     4         249 792 119.9Mbps        40 492  19.4Mbps                 209 300 100.4Mbps 5.54ms
-TOT    1       2 307 840 190.7Mbps       512 347  40.9Mbps              1 795 493 149.7Mbps 37.5us
+     5         249 791 119.8Mbps        46 736  22.4Mbps                 203 055  97.4Mbps 5.41ms
-TOT    2       1 980 927 163.7Mbps       763 490  61.0Mbps              1 217 437 102.6Mbps 2.94ms
+     TOT     1 498 751 735.3Mbps       '''516 513''' 247.9Mbps                 982 238 487.4Mbps 3.15ms
-TOT    3       2 310 745 191.0Mbps       509 504  40.7Mbps              1 801 241 150.2Mbps 19.1us
-TOT    4       2 360 869 188.8Mbps       247 539  19.8Mbps              2 113 330 169.0Mbps 2.74ms
-TOT    5       2 360 743 188.8Mbps       261 714  20.9Mbps              2 099 029 167.9Mbps 963us
+We can now add more firewall rules, queues, and any other configuration and see how many routers can actually handle.
-TOT    TOT    13 291 128 1086.1...     3 062 328 244.9Mbps             10 228 800 125.3Mbps 19.1us
-</pre>
-We can now add more firewall rules, queues and any other configuration and see how much router can actually handle.
+'''Lets add some firewall rules'''
+We will take the customer protection rules from the [[Manual:IP/Firewall/Filter#Customer_protection | manual]]
-Lets add some firewall rules from the [[Manual:IP/Firewall/Filter#Customer_protection | manual]]
 Start by adding default rules that should present on any firewall:
 <pre>
 /ip firewall filter
 add chain=forward protocol=tcp connection-state=invalid \
- 	action=drop comment="drop invalid connections"
+	action=drop comment="drop invalid connections"
 add chain=forward connection-state=established action=accept \
- 	comment="allow already established connections"
+	comment="allow already established connections"
 add chain=forward connection-state=related action=accept \
- 	comment="allow related connections"
+	comment="allow related connections"
 </pre>
-We get approximately 40Mbps less
+We get approximately '''18%''' less packets
-<pre>
-TOT    TOT    13 926 377 1138.0...     2 586 802 206.9Mbps                  11 339 575 215.3Mbps 41.6us
+     TOT     1 492 520 732.3Mbps       '''435 546''' 209.0Mbps               1 056 974 523.2Mbps 3.08ms
-</pre>
-Now add more rules from the manual to see how count of firewall rules influence performance of the board
+Now add more rules from the manual to see how the count of firewall rules affects the performance of the board
 <pre>
 /ip firewall filter
@@ Line 165: / Line 177: @@
 add chain=icmp protocol=icmp icmp-options=11:0 action=accept \
   	comment="allow time exceed"
 add chain=icmp protocol=icmp icmp-options=12:0 action=accept \
   	comment="allow parameter bad"
 add chain=icmp action=drop comment="deny all other types"
 </pre>
-<pre>
-TOT    TOT    16 309 055 1142.4...     2 978 839 204.2Mbps                  13 330 216 324.5Mbps 53.3us
+     TOT     1 500 908 736.4Mbps       '''424 197''' 203.6Mbps               1 076 711 532.8Mbps 4.07ms
-</pre>
 There are almost no performance changes.
-You can add further any amount of rules and see that there are minimum influence on performance of the router.
+You can add any amount of rules and see that there is only a small influence on the performance of the router.
@@ Line 181: / Line 193: @@
-Do the same with different packet sizes:
+'''Perform the same test with different packet sizes:'''
 <pre>
 /tool traffic-generator
@@ Line 189: / Line 201: @@
 </pre>
+If we run the test with 1518 packet size then max throughput will be only '''2.9Gbps''' This is because wire speed of all interfaces are reached.
+We will need to add one more port to our test and add streams.
+Connect '''ether12''' to '''ether12''' and proceed with configuration
+On DUT:
+<pre>
+/ip address
+add address=192.168.89.254/24 interface=ether12
+</pre>
+On TrafficGen
+<pre>
+/ip address
+add address=192.168.89.254/24 interface=ether12
+/tool traffic-generator packet-template
+add header-stack=mac,ip,udp ip-dst=192.168.89.254/32 ip-gateway=192.168.86.254 ip-src=192.168.86.1/32 name=\
+    r14
+add header-stack=mac,ip,udp ip-dst=192.168.89.254/32 ip-gateway=192.168.87.254 ip-src=192.168.87.1/32 name=\
+    r24
+add header-stack=mac,ip,udp ip-dst=192.168.89.254/32 ip-gateway=192.168.88.254 ip-src=192.168.88.1/32 name=\
+    r34
+add header-stack=mac,ip,udp ip-dst=192.168.86.1/32 ip-gateway=192.168.89.254 ip-src=192.168.89.254/32 name=\
+    r41
+add header-stack=mac,ip,udp ip-dst=192.168.87.1/32 ip-gateway=192.168.89.254 ip-src=192.168.89.254/32 name=\
+    r42
+add header-stack=mac,ip,udp ip-dst=192.168.88.1/32 ip-gateway=192.168.89.254 ip-src=192.168.89.254/32 name=\
+    r43
+</pre>
+And now run the test:
+<pre>
+/tool traffic-generator quick tx-template=r12,r13,r14,r21,r23,r24,r31,r32,r34,r41,r42,r43 \
+ packet-size=1514 mbps=350
+</pre>
+     6          23 472 284.2Mbps        23 328 282.5Mbps                     144 1744.1... 3.22ms
+     7          28 890 349.9Mbps        28 741 348.1Mbps                     149 1804.6... 1.74ms
+     8          28 889 349.9Mbps        26 870 325.4Mbps                   2 019  24.4Mbps 984us
+     9          23 455 284.0Mbps        23 083 279.5Mbps                     372   4.5Mbps 866us
+     10         28 876 349.7Mbps        28 709 347.7Mbps                     167   2.0Mbps 922us
+     11         28 875 349.7Mbps        27 277 330.3Mbps                   1 598  19.3Mbps 3.33ms
+     TOT       323 389   3.9Gbps       311 743   '''3.7Gbps'''                  11 646 143.6Mbps 341us
+As you can see we get '''3.7Gbps'''.
+And with all firewalls enabled from previous tests we get 2.8Gbps which is approximately '''30%''' slower:
+     TOT       275 405   3.3Gbps       238 143   '''2.8Gbps'''                 37 262 453.9Mbps 1.57ms
 {{Note | mind that speed in quick mode is specified per stream, so if you have two streams per port, you need to send 1/2 of traffic per stream}}
@@ Line 206: / Line 271: @@
 <pre>
 /ip address
-add address=1.1.1.1/24 interface=ether1 network=1.1.1.0
+add address=192.168.86.1/24 interface=ether1 network=192.168.86.0
-add address=2.2.2.2/24 interface=ether6 network=2.2.2.0
+add address=192.168.87.1/24 interface=ether6 network=192.168.87.0
-add address=3.3.3.3/24 interface=ether11 network=3.3.3.0
+add address=192.168.88.1/24 interface=ether11 network=192.168.88.0
 /tool traffic-generator packet-template
-add header-stack=mac,ip,udp ip-src=1.1.1.1/32 ip-dst=2.2.2.2/32 name=b12
+add header-stack=mac,ip,udp ip-src=192.168.86.1/32 ip-dst=192.168.87.1/32 name=b12
-add header-stack=mac,ip,udp ip-src=1.1.1.1/32 ip-dst=3.3.3.3/32 name=b13
+add header-stack=mac,ip,udp ip-src=192.168.86.1/32 ip-dst=192.168.88.1/32 name=b13
-add header-stack=mac,ip,udp ip-src=2.2.2.2/32 ip-dst=1.1.1.1/32 name=b21
+add header-stack=mac,ip,udp ip-src=192.168.87.1/32 ip-dst=192.168.86.1/32 name=b21
-add header-stack=mac,ip,udp ip-src=2.2.2.2/32 ip-dst=3.3.3.3/32 name=b23
+add header-stack=mac,ip,udp ip-src=192.168.87.1/32 ip-dst=192.168.88.1/32 name=b23
-add header-stack=mac,ip,udp ip-src=3.3.3.3/32 ip-dst=1.1.1.1/32 name=b31
+add header-stack=mac,ip,udp ip-src=192.168.88.1/32 ip-dst=192.168.86.1/32 name=b31
-add header-stack=mac,ip,udp ip-src=3.3.3.3/32 ip-dst=2.2.2.2/32 name=b32
+add header-stack=mac,ip,udp ip-src=192.168.88.1/32 ip-dst=192.168.87.1/32 name=b32
 </pre>
@@ Line 229: / Line 294: @@
 quick tx-template=b12,b13,b21,b23,b31,b32 packet-size=1514 mbps=500
 </pre>
+With small packets, we get approximately 1.4 mil packets per second
+    0         195 659  97.0Mbps       195 640  93.9Mbps                      19   3.1Mbps 22us
+    1         236 906 117.5Mbps       221 901 106.5Mbps                  15 005  10.9Mbps 18.7us
+    2         202 678 100.5Mbps       202 678  97.2Mbps                       0   3.2Mbps 18.7us
+    3         238 750 118.4Mbps       231 348 111.0Mbps                   7 402   7.3Mbps 12.1us
+    4         263 906 126.6Mbps       256 146 122.9Mbps                   7 760   3.7Mbps 23.9us
+    5         263 906 126.6Mbps       256 030 122.8Mbps                   7 876   3.7Mbps 14.3us
+    TOT     1 401 805 686.8Mbps     '''1 363 743''' 654.5Mbps                  38 062  32.2Mbps 12.1us
+With 1518 byte packets we will get wire-speed maximum
+     TOT       243 587   2.9Gbps       241 695   2.9Gbps                   1 892  25.5Mbps 1.04ms
+So we will need to use ether12 and add a few more streams just like in the routing test.
+==CCR1036-8G-2S+ Test setup==
+CCR1036 series routers have very powerful CPU, So the test will use all ports. As shown in the diagram below
+[[File:CCR1036-8splus-diagram.png | center |700px]]
+The test will be separated in units, each unit will hold a pair of interfaces. Here is the list of units and what medium is used:
+*sfp-sfpplus1 will pass data to sfp-sfpplus2 (direct attach copper 10GBit)
+*ether1 to ether2 (cat5e)
+*ether3 to ether4 (cat5e)
+*ether5 to ether6 (cat5e)
+*ether7 to ether8 (cat5e)
+For testing we will require 2 other CCR1036 as passing data through is not as intensive as generating traffic and gathering statistical data.
+{{Note| For a test with large packets only one additional CCR1036 is required for use with the traffic-generator tool. All the configuration examples will not interfere with each other and can be imported on one testing router}}
+Connect TR1  (Trafic-generator Router 1) to DUT:
+* SFP/SFP+1 to SFP/SFP+1 Direct attach 10GBit cable (10Gbit optical SFP interfaces cable used instead)
+* SFP/SFP+2 to SFP/SFP+2 Direct attach 10GBit cable (10Gbit optical SFP interfaces cable used instead)
+Connect TR2 to DUT:
+*ether1 to ether1
+*ether2 to ether2
+*ether3 to ether3
+*ether4 to ether4
+*ether5 to ether5
+*ether6 to ether6
+*ether7 to ether7
+*ether8 to ether8
+====Defaults, routing====
+The default configuration of DUT is with enabled fast-path
+======DUT configuration======
+  /ip address
+  add address=10.0.100.1/24 interface=sfp-sfpplus1 network=10.0.100.0
+  add address=10.0.101.1/24 interface=sfp-sfpplus2 network=10.0.101.0
+  add address=10.0.110.1/24 interface=ether1 network=10.0.110.0
+  add address=10.0.111.1/24 interface=ether2 network=10.0.111.0
+  add address=10.0.112.1/24 interface=ether3 network=10.0.112.0
+  add address=10.0.113.1/24 interface=ether4 network=10.0.113.0
+  add address=10.0.114.1/24 interface=ether5 network=10.0.114.0
+  add address=10.0.115.1/24 interface=ether6 network=10.0.115.0
+  add address=10.0.116.1/24 interface=ether7 network=10.0.116.0
+  add address=10.0.117.1/24 interface=ether8 network=10.0.117.0
+======TR1 configuration (10Gbit interfaces)======
+  /ip address
+  add address=10.0.100.2/24 interface=sfp-sfpplus1 network=10.0.100.0
+  add address=10.0.101.2/24 interface=sfp-sfpplus2 network=10.0.101.0
+  /tool traffic-generator packet-template
+  add header-stack=mac,ip,udp interface=sfp-sfpplus1 ip-dst=10.0.101.2 ip-gateway=10.0.100.1 name=pt10
+  add header-stack=mac,ip,udp interface=sfp-sfpplus2 ip-dst=10.0.100.2 ip-gateway=10.0.101.1 name=pt11
+  /tool traffic-generator stream
+  add id=0 mbps=6000 name=str10 packet-size=60 tx-template=pt10
+  add id=1 mbps=6000 name=str11 packet-size=60 tx-template=pt11
+======TR2 configuration (1Gbit interfaces)======
+  /ip address
+  add address=10.0.110.2/24 interface=ether1 network=10.0.110.0
+  add address=10.0.111.2/24 interface=ether2 network=10.0.111.0
+  add address=10.0.112.2/24 interface=ether3 network=10.0.112.0
+  add address=10.0.113.2/24 interface=ether4 network=10.0.113.0
+  add address=10.0.114.2/24 interface=ether5 network=10.0.114.0
+  add address=10.0.115.2/24 interface=ether6 network=10.0.115.0
+  add address=10.0.116.2/24 interface=ether7 network=10.0.116.0
+  add address=10.0.117.2/24 interface=ether8 network=10.0.117.0
+  /tool traffic-generator packet-template
+  add header-stack=mac,ip,udp ip-dst=10.0.111.2 ip-gateway=10.0.110.1 name=pt1
+  add header-stack=mac,ip,udp ip-dst=10.0.110.2 ip-gateway=10.0.111.1 name=pt2
+  add header-stack=mac,ip,udp ip-dst=10.0.113.2 ip-gateway=10.0.112.1 name=pt3
+  add header-stack=mac,ip,udp ip-dst=10.0.112.2 ip-gateway=10.0.113.1 name=pt4
+  add header-stack=mac,ip,udp ip-dst=10.0.115.2 ip-gateway=10.0.114.1 name=pt5
+  add header-stack=mac,ip,udp ip-dst=10.0.114.2 ip-gateway=10.0.115.1 name=pt6
+  add header-stack=mac,ip,udp ip-dst=10.0.117.2 ip-gateway=10.0.116.1 name=pt7
+  add header-stack=mac,ip,udp ip-dst=10.0.116.2 ip-gateway=10.0.117.1 name=pt8
+  /tool traffic-generator stream
+  add id=0 mbps=700 name=str0 packet-size=60 tx-template=pt0
+  add id=1 mbps=700 name=str1 packet-size=60 tx-template=pt1
+  add id=2 mbps=700 name=str3 packet-size=60 tx-template=pt2
+  add id=3 mbps=700 name=str4 packet-size=60 tx-template=pt3
+  add id=4 mbps=700 name=str5 packet-size=60 tx-template=pt4
+  add id=5 mbps=700 name=str6 packet-size=60 tx-template=pt5
+  add id=6 mbps=700 name=str7 packet-size=60 tx-template=pt6
+  add id=7 mbps=700 name=str8 packet-size=60 tx-template=pt7
+======Running test======
+To run tests use
+ON TR1
+  /tool traffic-generator quick packet-size=60 mbps=6000
+On TR2
+  /tool traffic-generator quick packet-size=60 mbps=700
+Profile for medium-sized packets:
+ON TR1
+  /tool traffic-generator quick packet-size=508 mbps=10000
+On TR2
+  /tool traffic-generator quick packet-size=508 mbps=1000
+Profile for large packets:
+ON TR1
+  /tool traffic-generator quick packet-size=1514 mbps=10000
+On TR2
+  /tool traffic-generator quick packet-size=1514 mbps=1000
 ==See More==
 * [[M:Tools/Traffic_Generator | Traffic Generator Manual]]
+* [[M:Fast_Path | Fast Path]]

Manual:Performance Testing with Traffic Generator: Difference between revisions

Latest revision as of 13:21, 1 June 2020

Summary

RB1100AHx2 Test setup

Connecting the routers

Routing Performance Testing

DUT Config

Traffic Generator Config

Running Tests

Bridging Performance Testing

DUT Config

Traffic Generator Config

Running Tests

CCR1036-8G-2S+ Test setup

Defaults, routing

DUT configuration

TR1 configuration (10Gbit interfaces)

TR2 configuration (1Gbit interfaces)

Running test

See More

Navigation menu

Search