Manual:Layer2 misconfiguration
Applies to RouterOS: v6.41 +
Introduction
There are certain configuration that are known to have major flaws by design and should be avoided by all means possible. Misconfigured Layer2 can sometimes cause hard to detect network errors, random performance drops, certain segments of a network to be unreachable, certain networking services to be malfunctioning or a complete network failure. This page will contain some common and not so very common configurations that will cause issues in your network.
Bridges on a single switch chip
Consider the following scenario, you have a device with a built-in switch chip and you need to isolate certain ports from each other, for this reason you have created multiple bridges and enabled hardware offloading on them. Since each bridge is located on a different Layer2 domain, then Layer2 frames will not be forwarded between these bridges, as a result ports in each bridge are isolated from other ports in a different bridge.
Configuration
/interface bridge add name=bridge1 add name=bridge2 /interface bridge port add bridge=bridge1 interface=ether1 add bridge=bridge1 interface=ether2 add bridge=bridge2 interface=ether3 add bridge=bridge2 interface=ether4
Problem
After a simple performance test you might notice that one bridge is capable of forwarding traffic at wire-speed while the second, third, ... bridge is not able to forward as much data as the first bridge. Another symptom might be that there exists a huge latency for packets that need to be routed. After a quick inspection you might notice that the CPU is always at full load, this is because hardware offloading is not available on all bridges, but is available only on one bridge. By checking the hardware offloading status you will notice that only one bridge has it active:
[admin@MikroTik] > /interface bridge port print Flags: X - disabled, I - inactive, D - dynamic, H - hw-offload # INTERFACE BRIDGE HW 0 H ether1 bridge1 yes 1 H ether2 bridge1 yes 2 ether3 bridge2 yes 3 ether4 bridge2 yes
The reason why only one bridge has the hardware offloading flag available is because the device does not support port isolation. If port isolation is not supported, then only one bridge will be able to offload the traffic to the switch chip.
Solution
Not all device devices support port isolation, currently only CRS1xx/CRS2xx series devices support it, other devices will have to use the CPU to forward the packets on other bridges. This is usually a hardware limitation and a different device might be required. Bridge split horizon parameter is a software feature that disables hardware offloading and when using bridge filter rules you need to enable forward all packets to the CPU, which requires the hardware offloading to be disabled. You can control which bridge will be hardware offloaded with the hw=yes flag and by setting hw=no to other bridges, for example:
/interface bridge port set [find where bridge=bridge1] hw=no /interface bridge port set [find where bridge=bridge2] hw=yes
Sometimes it is possible to restructure a network topology to use VLANs, which is the proper way to isolate Layer2 networks.
VLAN on a bridge in a bridge
Setup
Consider the following scenario, you have a set of interfaces (don't have to be physical interfaces) and you want all of them to be in the same Layer2 segment, the solution is to add them to a single bridge, but you require that traffic from one port tags all traffic into a certain VLAN. This can be done by creating a VLAN interface on top of the bridge interface and by creating a separate bridge that contains this newly created VLAN interface and the interface, which will send out tagged traffic. Network diagram can be found below:
Configuration
/interface bridge add name=bridge1 add name=bridge2 /interface vlan add interface=bridge1 name=VLAN vlan-id=99 /interface bridge port add bridge=bridge1 interface=ether1 add bridge=bridge1 interface=ether2 add bridge=bridge2 interface=VLAN add bridge=bridge2 interface=ether3
Problem
Packets coming from ether3 will be sent out tagged and traffic won't be flooded through ether1 and ether2, but if another port is added to bridge2, then traffic will be flooded. Similar issue arises when traffic needs to be sent from ether1 to ether3 since MAC learning is only possible between bridge ports and not interfaces that are created on top of the bridge interface. As a result unicast traffic will be flooded to ether2 and ether3. If a device behind ether3 is using (R)STP, then ether1 and ether2 will send out tagged BPDUs. Because of the broken MAC learning functionality and broken (R)STP this setup and configuration must be avoided.
Solution
Use bridge VLAN filtering. The proper way to tag traffic is to assign a VLAN ID whenever traffic enters a bridge, this behaviour can easily be achieved by specifying PVID value for a bridge port and specifying which ports are tagged (trunk) ports and which are untagged (access) ports. Below is an example how such setup should have been configured:
/interface bridge add name=bridge vlan-filtering=yes /interface bridge port add bridge=bridge interface=ether1 add bridge=bridge interface=ether2 add bridge=bridge interface=ether3 pvid=99 /interface bridge vlan add bridge=bridge tagged=ether1 untagged=ether3 vlan-ids=99
Warning: By enabling vlan-filtering you will be filtering out traffic destined to the CPU, before enabling VLAN filtering you should make sure that you set up a Management port