Manual:Maximum Transmission Unit on RouterBoards: Difference between revisions

From MikroTik Wiki
Jump to navigation Jump to search
(139 intermediate revisions by 9 users not shown)
Line 16: Line 16:
It is common now for Ethernet interfaces to support physical MTU above standard, but this can not be taken for granted. Abilities of other network equipment must be taken into account as well - for example, if 2 routers with Ethernet interfaces supporting physical MTU 1526 are connected through Ethernet switch, in order to successfully implement some application that will produce this big Ethernet frames, switch must also support forwarding such frames.
It is common now for Ethernet interfaces to support physical MTU above standard, but this can not be taken for granted. Abilities of other network equipment must be taken into account as well - for example, if 2 routers with Ethernet interfaces supporting physical MTU 1526 are connected through Ethernet switch, in order to successfully implement some application that will produce this big Ethernet frames, switch must also support forwarding such frames.


=MTU on RouterBoards=
=MTU on RouterOS=
 
[[Image:MTU general explanation.png|left|300px|Different types of MTU]]
 
 
Mikrotik RouterOS recognizes several types of MTU:
 
*IP/Layer-3/L3 MTU
 
*MPLS/Layer-2.5/L2.5 MTU
 
*MAC/Layer-2/L2 MTU
 
*Full frame MTU
 
 
 
 
==Full frame MTU==
 
Full frame MTU indicates the actual size of the frame that are sent by particular interface. Frame Checksum is not included as it is removed by Ethernet driver as soon as frame reach its destination.
 
==MAC/Layer-2/L2 MTU  ==
 
L2MTU indicates the maximum size of the frame without MAC header that can be sent by this interface.
 
Starting from the RouterOS v3.25 L2MTU values can be seen in "/interface" menu.  L2MTU support is added for all Routerboard related Ethernet interfaces, VLANs, Bridge, VPLS and wireless interfaces. Some of them support configuration of L2MTU value. All other Ethernet interfaces might indicate L2MTU only if the chip set is the same as Routerboard Ethernets.
 
This will allow users to check if desired setup is possible. Users will be able to utilize additional bytes for VLAN and MPLS tags, or simple increase of interface MTU to get rid of the some unnecessary fragmentation.
 
This table shows max-l2mtu supported by Mikrotik RouterBoards (Starting from the RouterOS v5.3 also available in "/interface print" menu as value of read-only "max-l2mtu" option):


This table shows how many additional bytes is supported by Mikrotik RouterBoards outside of standard (IEEE) Ethernet MTU:
<table class="styled_table">
<table class="styled_table">
<tr>
<tr>
   <th width="100">RouterBoard</th>
   <th width="100">Model name</th>
   <th width="50">ether1</th>
   <th width="400">MTU description</th>
   <th width="50">ether2</th>
</tr>
   <th width="50">ether3</th>
<tr>
   <th width="50">ether4</th>
  <td ><b>RB Groove series</b></td>
   <th width="50">ether5</th>
  <td >ether1:2028</td>
   <th width="50">ether6</th>
</tr>
   <th width="50">ether7</th>
<tr>
   <th width="50">ether8</th>
  <td ><b>RB QRT series</b></td>
   <th width="50">ether9</th>
  <td >ether1:4076</td>
</tr>
<tr>
  <td ><b>RB Metal series</b></td>
  <td >ether1:2028</td>
</tr>
<tr>
  <td ><b>RB SXT series, RB LHG, RB LDF, PL6411-2nD, PL7411-2nD, wAP R-2nD, RB912R-2nD-LTm (LtAP mini)</b></td>
  <td >ether1:2028</td>
</tr>
<tr>
  <td ><b>RB SXT Lite series</b></td>
  <td >ether1:2028</td>
</tr>
<tr>
<tr>
  <td ><b>RB SXT G series, RB DynaDish, wAP ac</b></td>
  <td >ether1:4076</td>
</tr>
<tr>
  <td ><b>RB OmniTik series</b></td>
  <td >ether1:4076; ether2-ether5:2028</td>
</tr>
<tr>
  <td ><b>RB OmniTik ac series</b></td>
  <td >ether1-ether5:4074</td>
</tr>
<tr>
  <td ><b>RB mAP, RB mAP lite, RB cAP, RB wAP</b></td>
  <td >ether1-ether2:2028</td>
</tr>
<tr>
  <td ><b>RB750</b></td>
  <td >ether1:4076; ether2-ether5:2028</td>
</tr>
<tr>
  <td ><b>RB750r2, RB750P-PBr2, RB750UPr2</b></td>
  <td >ether1-ether5:2028</td>
</tr>
<tr>
  <td ><b>RB750UP</b></td>
  <td >ether1:4076; ether2-ether5:2028</td>
</tr>
<tr>
  <td ><b>RB751U-2HnD</b></td>
  <td >ether1:4076; ether2-ether5:2028</td>
</tr>
<tr>
  <td ><b>RB951-2n</b></td>
   <td >ether1:4076; ether2-ether5:2028</td>
</tr>
<tr>
  <td ><b>RB941-2nD, RB951Ui/RB952Ui series</b></td>
  <td >ether1-ether5:2028</td>
</tr>
<tr>
  <td ><b>RB750GL, RB750Gr2</b></td>
  <td >ether1-ether5:4074</td>
</tr>
<tr>
  <td ><b>RB750Gr3</b></td>
  <td >ether1-ether5:2026</td>
</tr>
<tr>
  <td ><b>RB751G-2HnD</b></td>
  <td >ether1-ether5:4074</td>
</tr>
<tr>
  <td ><b>RB951G-2HnD</b></td>
  <td >ether1-ether5:4074</td>
</tr>
<tr>
  <td ><b>RB962UiGS, RB960PGS</b></td>
  <td >ether1-ether5:4074; sfp1:4076</td>
</tr>
<tr>
  <td ><b>RB1100Hx2</b></td>
  <td >ether1-ether10:9498; ether11:9500; ether12-ether13:9116</td>
</tr>
<tr>
  <td ><b>RB1100AHx2</b></td>
  <td >ether1-ether10:9498; ether11:9500; ether12-ether13:9116</td>
</tr>
<tr>
  <td ><b>RB4011iGS+ series</b></td>
  <td >ether1-ether10:9578; sfp-sfpplus1:9982</td>
</tr>
<tr>
   <td ><b>CCR1009 series</b></td>
   <td >ether1-ether4:10224; ether5-ether8:10226; sfp1:10226; sfp-sfpplus1:10226</td>
</tr>
<tr>
  <td ><b>CCR1016 series</b></td>
  <td >ether1-ether12:10226; sfp1-sfp12:10226; sfp-sfpplus1:10226</td>
</tr>
<tr>
  <td ><b>CCR1036 series</b></td>
  <td >ether1-ether12:10226; sfp1-sfp4:10226; sfp-sfpplus1-sfp-sfpplus2:10226</td>
</tr>
<tr>
  <td ><b>CCR1072 series</b></td>
  <td >ether1:9116; sfp-sfpplus1-sfp-sfpplus8:10226</td>
</tr>
<tr>
  <td ><b>CRS109-8G-1S</b></td>
  <td >ether1-ether8:4064; sfp1:4064</td>
</tr>
<tr>
  <td ><b>CRS125-24G-1S</b></td>
  <td >ether1-ether24:4064; sfp1:4064</td>
</tr>
<tr>
  <td ><b>CRS112-8G-4S</b></td>
  <td >ether1-ether8:9204; sfp9-sfp12:9204</td>
</tr>
<tr>
   <td><b>CRS106-1C-5S </b></td>
   <td>sfp1-sfp5:9204; combo1:9204</td>
</tr>
<tr>
   <td><b>CRS210-8G-2S+</b></td>
   <td>ether1-ether8:9204; sfp-sfpplus1:9204; sfpplus2:9204</td>
</tr>
<tr>
  <td><b>CRS212-1G-10S-1S+</b></td>
  <td>ether1:9204; sfp1-sfp10:9204; sfpplus1:9204</td>
</tr>
<tr>
  <td><b>CRS226-24G-2S+</b></td>
  <td>ether1-ether24:9204; sfp-sfpplus1:9204; sfpplus2:9204</td>
</tr>
<tr>
  <td><b>CRS326-24G-2S+, CSS326-24G-2S+</b></td>
  <td>ether1-ether24:10218; sfp-sfpplus1:10218; sfpplus2:10218</td>
</tr>
<tr>
  <td><b>CRS317-1G-16S+</b></td>
  <td>ether1:10218; sfp-sfpplus1-sfp-sfpplus16:10218</td>
</tr>
<tr>
  <td><b>CRS328-24P-4S+</b></td>
  <td>ether1-ether24:10218; sfp-sfpplus1-sfp-sfpplus4:10218</td>
</tr>
<tr>
  <td><b>CRS328-4C-20S-4S+</b></td>
  <td>combo1-combo4:10218; sfp1-sfp20:10218; sfp-sfpplus1-sfp-sfpplus4:10218</td>
</tr>
<tr>
  <td><b>CRS305-1G-4S+</b></td>
  <td>ether1:10218; sfp-sfpplus1-sfp-sfpplus4:10218</td>
</tr>
<tr>
  <td><b>CRS309-1G-8S+</b></td>
  <td>ether1:10218; sfp-sfpplus1-sfp-sfpplus8:10218</td>
</tr>
<tr>
  <td><b>CRS312-4C+8XG</b></td>
   <td>combo1-combo4:10218; ether1-ether8:10218; ether9:2028</td>
</tr>
<tr>
  <td><b>CRS326-24S+2Q+</b></td>
  <td>sfp-sfpplus1-sfp-sfpplus24:10218; qsfpplus1-1-qsfpplus2-4:10218; ether1:2028</td>
</tr>
<tr>
  <td><b>D52G-5HacD2HnD (hAP ac&#178;)</b></td>
  <td>ether1-ether5:9124</td>
</tr>
<tr>
  <td><b>cAP ac</b></td>
  <td>ether1-ether2:9124</td>
</tr>
<tr>
<tr>
  <td><b>wAP60G</b></td>
  <td>ether1:9124</td>
</tr>
<tr>
  <td><b>Cube Lite60</b></td>
  <td>ether1:2028</td>
</tr>
<tr>
  <td><b>RB260GS series, CSS106-5G-1S, CSS106-1G-4P-1S</b></td>
  <td>ether1-ether5:9198; sfp1:9198</td>
</tr>
<tr>
  <td><b>RB FTC</b></td>
  <td>ether1:4046; sfp1:4046</td>
</tr>
<tr>
  <td><b>RBM33G</b></td>
  <td>ether1-ether3:2026</td>
</tr>
<tr>
  <td><b>RBM11G</b></td>
  <td>ether1:2026</td>
</tr>
<tr>
  <td><b>RB760iGS</b></td>
  <td>ether1-ether5:2026; sfp1:2026</td>
</tr>
<tr>
  <td ><b>RB411 series</b></td>
  <td >ether1:1526</td>
</tr>
</tr>
<tr>
<tr>
   <td ><b>RB493, RB493AH</b></td>
   <td ><b>RB433 series</b></td>
   <td >+22</td>
   <td >ether1:1526; ether2-ether3:1522</td>
  <td >+18</td>
  <td >+18</td>
  <td >+18</td>
  <td >+18</td>
  <td >+18</td> 
  <td >+18</td>
  <td >+18</td>
  <td >+18</td>
</tr>
</tr>
<tr>
<tr>
   <td ><b>RB450</b></td>
   <td ><b>RB450</b></td>
   <td >+22</td>
   <td >ether1:1526; ether2-ether5:1522</td>
   <td >+18</td>
</tr>
   <td >+18</td>
<tr>
   <td >+18</td>
  <td ><b>RB450Gx4</b></td>
   <td >+18</td>
  <td >ether1-ether5:9214</td>
</tr>
<tr>
  <td ><b>RB493 series</b></td>
  <td >ether1:1526; ether2-ether9:1522</td>
</tr>
<tr>
  <td><b>RB411GL</b></td>
  <td>ether1:1520</td>
</tr>
<tr>
  <td><b>RB433GL</b></td>
  <td>ether1-ether3:1520</td>
</tr>
<tr>
  <td><b>RB435G</b></td>
  <td>ether1-ether3:1520</td>
</tr>
<tr>
  <td><b>RB450G</b></td>
  <td>ether1-ether5:1520</td>
</tr>
<tr>
  <td><b>RB493G</b></td>
  <td>ether1-ether9:1520</td>
</tr>
<tr>
  <td ><b>RB711 series</b></td>
  <td >ether1:2028</td>
</tr>
<tr>
  <td ><b>RB711G series</b></td>
  <td >ether1:4076</td>
</tr>
<tr>
  <td ><b>RB800</b></td>
   <td >ether1-ether2:9500; ether3:9116</td>
</tr>
<tr>
   <td ><b>RB850Gx2</b></td>
   <td >ether1-ether5:1580</td>
</tr>
<tr>
  <td ><b>RB911G</b></td>
  <td >ether1:4076</td>
</tr>
<tr>
  <td ><b>RB912UAG</b></td>
  <td >ether1:4076</td>
</tr>
<tr>
  <td ><b>RB921UAGS, RB922UAGS</b></td>
  <td >ether1:4076; sfp1:4076</td>
</tr>
<tr>
  <td ><b>RB953GS</b></td>
   <td >ether1-ether2:4074; sfp1:4074; sfp2:4076</td>
</tr>
</tr>
<tr>
<tr>
   <td ><b>RB433, RB433AH</b></td>
   <td ><b>RB2011 series</b></td>
   <td >+22</td>
   <td >ether1-ether5:4074; ether6-ether10:2028; sfp1:4074</td>
  <td >+18</td>
  <td >+18</td>
</tr>
</tr>
<tr>
<tr>
   <td ><b>RB411, RB411A, RB411AH</b></td>
   <td ><b>RB3011 series</b></td>
   <td >+22</td>
   <td >ether1-ether5:8156; ether6-ether10:8156; sfp1:8158</td>
</tr>
</tr>
<tr>
<tr>
   <td ><b>CrossRoads</b></td>
   <td ><b>RB44Ge</b></td>
   <td >+100</td>
   <td >ether1-ether4:9116</td>
</tr>
</table>
 
'''Old Products'''
<table class="styled_table">
<tr>
  <th width="100">RouterBoard</th>
  <th width="400">MTU description</th>
</tr>
<tr>
  <td ><b>RB600 series</b></td>
  <td >ether1-ether3:9500</td>
</tr>
</tr>
<tr>
<tr>
   <td ><b>RB1000</b></td>
   <td ><b>RB1000</b></td>
   <td >+8000</td>
   <td >ether1-ether4:9500</td>
   <td >+8000</td>
</tr>
   <td >+8000</td>
<tr>
   <td >+8000</td>
   <td ><b>RB1100</b></td>
   <td >ether1-ether10:9498; ether11-ether13:9116</td>
</tr>
<tr>
  <td ><b>RB1100AH</b></td>
  <td >ether1-ether10:9498; ether11:9500, ether12-ether13:9116</td>
</tr>
<tr>
  <td ><b>RB1200</b></td>
  <td >ether1-ether5:4078, ether6-ether8:4080, ether9-ether10:9116</td>
</tr>
<tr>
  <td ><b>RB750 (old revision)</b></td>
   <td >ether1:1526; ether2-ether5:1522</td>
</tr>
</tr>
<tr>
<tr>
   <td ><b>RB600, RB600A</b></td>
   <td ><b>RB750G</b></td>
   <td >+8000</td>
  <td >ether1-ether5:1524</td>
   <td >+8000</td>
</tr>
   <td >+7500</td>
<tr>
  <td ><b>RB333</b></td>
  <td >ether1-ether3:1632</td>
</tr>
<tr>
  <td ><b>RB1xx</b></td>
  <td >ether1-ether5:1518; ether6-ether9:1514</td>
</tr>
<tr>
  <td ><b>RB532, CrossRoads</b></td>
   <td >ether1-ether3:1600</td>
</tr>
<tr>
  <td ><b>RB44G</b></td>
   <td >ether1-ether4:7200</td>
</tr>
<tr>
  <td ><b>RB44GV</b></td>
  <td >ether1-ether4:9000</td>
</tr>
<tr>
  <td ><b>RB250GS</b></td>
   <td >ether1-ether5:9198</td>
</tr>
</tr>
</table>
</table>


So if table indicate "+22" - interface has Layer 2 MTU 14+22+1500=1534 bytes, and can handle 22 bytes (over standard Ethernet MTU of 1500) that can be utilized for VLAN tags, MPLS tags, Increase of layer-3 MTU


All wireless interfaces in RouterOS (including Nstreme2) Layer-2 1600 byte Physical MTU, so you have spare '''+86''' bytes for VLAN tags, MPLS tags, Increase of layer-3 MTU
All wireless interfaces in RouterOS (including Nstreme2) support  2290 byte L2MTU.


=MTU in RouterOS=
{{Warning | L2MTU configuration changes evoke all interface reload (link down - link up) due to necessary internal processes.<br />
It is recommended to configure L2MTU with caution by keeping in mind that it can cause short interruption with connected devices.}}


==IP/Layer-3 MTU==
==MPLS/Layer-2.5/L2.5 MTU==


Configured as interface mtu setting (/interface <type> <name> set mtu=X). Specifies how big IP packets router is allowed to send out the particular interface.
Configured in "/mpls interface" menu, specifies maximal size of packet, including MPLS labels, that is allowed to send out by the particular interface (default is 1508).


If router receives IP packet of size 1500, but MTU for outgoing interface is set to 1400, router will either fragment the packet (if "Don't Fragment" bit is not set in IP header) or drop the packet and send ICMP "Need Fragmentation" error back to originator (this is essential for Path MTU Discovery to work).
Make sure that MPLS MTU is smaller or equal to L2MTU
 
Sometimes it can be bad idea to change IP MTU from its default 1500 bytes on router interfaces if complete path end-to-end is not in administrators control. Although IP fragmentation and end-to-end Path MTU Discovery is intended to handle this situation, if ICMP Need Fragmentation errors are filtered somewhere along the path, Path MTU Discovery will not work.
 
There are several features in MikroTik RouterOS that is can benefit from possibility to exceed standard MTU
 
==MPLS MTU ==
 
Configured in "/mpls interface" menu, specifies how big packets, including MPLS labels, router is allowed to send out the particular interface (default is 1508).
 
Note that when actually sending packet out the interface, link layer header is going to get attached, so it is only safe to configure MPLS MTU up to the value of physical MTU minus size of any link layer header(s) that is going to get applied (e.g. 14 in regular ethernet case, or 18 in case of one VLAN tag).


MPLS MTU affects packets depending on what action MPLS router is performing. It is strongly recommended that MPLS MTU is configured to the same value on all routers forming MPLS cloud because of effects MPLS MTU has on MPLS switched packets. This requirement means that all interfaces participating in MPLS cloud must be configured to the smallest MPLS MTU values among participating interfaces, therefore care must be taken to properly select hardware to be used.
MPLS MTU affects packets depending on what action MPLS router is performing. It is strongly recommended that MPLS MTU is configured to the same value on all routers forming MPLS cloud because of effects MPLS MTU has on MPLS switched packets. This requirement means that all interfaces participating in MPLS cloud must be configured to the smallest MPLS MTU values among participating interfaces, therefore care must be taken to properly select hardware to be used.
Line 109: Line 423:
If packet with labels included is bigger than MPLS MTU, MPLS tries to guess protocol that is carried inside MPLS frame.
If packet with labels included is bigger than MPLS MTU, MPLS tries to guess protocol that is carried inside MPLS frame.


If this is IP packet, MPLS produces ICMP Need Fragment error. This behaviour mimics IP protocol behaviour. Note that this ICMP error is not routed back to originator of packet but is switched towards end of LSP, so that egress router can route it back.
If this is IP packet, MPLS produces ICMP Need Fragment error. This behavior mimics IP protocol behavior. Note that this ICMP error is not routed back to originator of packet but is switched towards end of LSP, so that egress router can route it back.


If this is not IP packet, MPLS simply drops it, because it does not know how to interpret the contents of packet. This feature is very important in situations where MPLS applications such as VPLS are used (where frames that are MPLS tagged are not IP packets, but e.g. encapsulated ethernet frames as in case of VPLS) - if somewhere along the LSP MPLS MTU will be less than packet size prepared by ingress router, frames will simply get dropped.
If this is not IP packet, MPLS simply drops it, because it does not know how to interpret the contents of packet. This feature is very important in situations where MPLS applications such as VPLS are used (where frames that are MPLS tagged are not IP packets, but e.g. encapsulated Ethernet frames as in case of VPLS) - if somewhere along the LSP MPLS MTU will be less than packet size prepared by ingress router, frames will simply get dropped.


===IP ingress===
===IP ingress===
Line 119: Line 433:
===VPLS ingress===
===VPLS ingress===


When router encapsulates ethernet frame for forwarding over VPLS pseudowire, it checks if packet size with VPLS Control Word (4 bytes) and any necessary labels (usually 2 labels - 8 bytes), exceeds MPLS MTU of outgoing interface. If it does, VPLS fragments packet so that it honours MPLS MTU of outgoing interface. Packet is defragmented at egress point of VPLS pseudowire.
When router encapsulates Ethernet frame for forwarding over VPLS pseudowire, it checks if packet size with VPLS Control Word (4 bytes) and any necessary labels (usually 2 labels - 8 bytes), exceeds MPLS MTU of outgoing interface. If it does, VPLS fragments packet so that it honours MPLS MTU of outgoing interface. Packet is defragmented at egress point of VPLS pseudowire.
 
==IP/Layer-3/L3 MTU==


== MTU examples for common technologies ==
Configured as interface MTU setting (/interface <type> <name> set mtu=X). Specifies how big IP packets router is allowed to send out the particular interface.


Examples on how different technologies and their combinations relate to MTUs are given below. All examples are given assuming common standard 1500 byte IP packet size and that outgoing interface is ethernet. The total size of packet after all encapsulations must not exceed the physical MTU of outgoing interface.
If router receives IP packet of size 1500, but MTU for outgoing interface is set to 1400, router will either fragment the packet (if "Don't Fragment" bit is not set in IP header) or drop the packet and send ICMP "Need Fragmentation" error back to originator (this is essential for Path MTU Discovery to work).
 
Sometimes it can be bad idea to change IP MTU from its default 1500 bytes on router interfaces if complete path end-to-end is not in administrators control. Although IP fragmentation and end-to-end Path MTU Discovery is intended to handle this situation, if ICMP Need Fragmentation errors are filtered somewhere along the path, Path MTU Discovery will not work.
 
There are several features in MikroTik RouterOS that can benefit from possibility to exceed standard MTU
 
=Simple Examples=
 
In these examples we will take a look at frames entering and leaving router via Ethernet interfaces.


=== Simple Routing ===
=== Simple Routing ===


The image shows the packet MTU size for simple routing, packets size is not modified,
The image shows the packet MTU size for simple routing, packets size is not modified.




[[Image:MTUSimpleRouting.png]]
[[Image:MTUSimpleRouting.png | 740px]]
 


=== Routing with VLAN Encap ===
=== Routing with VLAN Encap ===


Each VLAN tag is 4 bytes long, VLAN tag is added by router. L2-MTU is increased by 4 bytes.


Each VLAN tag is 4 bytes long, VLAN tag is added by router. L2-MTU is increased by 4 bytes.


[[Image:MTUVLANENCAP.png | 740px]]


[[Image:MTUVLANENCAP.png]]


=== Simple MPLS with tags===
=== Simple MPLS with tags===
Line 145: Line 470:




[[Image:MTUMPLS2Tags.png]]
[[Image:MTUMPLS2Tags.png | 740px]]


=== Simple MPLS over VLAN ===


If simple MPLS packets (with 1 label) are forwarded across ethernet additionally encapsulated in Vlan:
=== VPLS Tunnel ===


 
Two MPLS labels are present, when remote endpoint is not directly attached. One MPLS label is used to get to remote endpoint, second label is used to identify VPLS tunnel.
[[Image:MTUMPLS1TAGVLAN.png]]




Note that MPLS MTU of VLAN interface matters in this case and it should be at least 1504.
[[Image:MTUVPLS.png | 740px]]




=== VPLS Tunnel ===
=L2MTU advanced example=


Two MPLS labels are present, when remote endpoint is not directly attached. One MPLS label is used to get to remote endpoint, second label is used to identify VPLS tunnel.
In this example we will take a closer look at required L2MTU of all Ethernet like interfaces including  Bridge, VLAN, VPLS interfaces.


In this setup we will have 3 routers:
* Q-in-Q router - this router will receive standard 1500 byte Ethernet frame and will add two VLAN tags to the packet. Then packet will be sent out via Ethernet network to the second router


[[Image:MTUVPLS.png]]
*VPLS router - this router will remove outer VLAN tag and will bridge packet with the remaining VLAN tag with VPLS tunnel. VPLS tunnel will take packet through the MPLS network to the third router.


*MPLS Edge router - will remove VPLS and VLAN tags and bridge packet to the client Ethernet network.




=== Q-in-Q over VPLS tunnel ===
[[Image:L2MTU example.png|740px]]




[[Image:MTU QinQVPLS.png]]


{{Cont}}


[[Category:Manual]]
[[Category:Manual]]
[[Category:Interface]]
[[Category:Case Studies]]
[[Category:Routerboard]]

Revision as of 14:00, 27 November 2019

Background

It is sole responsibility of administrator to configure MTUs such that intended services and applications can be successfully implemented in network. In other words - administrator must make sure that MTUs are configured in a way that packet sizes does not exceed the capabilities of network equipment.

Originally MTU was introduced because of the high error rates and low speed of communications. Fragmentation of the data stream gives ability to correct corruption errors only by resending corrupted fragment, not the whole stream. Also on low speed connections such as modems it can take too much time to send a big fragment, so in this case communication is possible only with smaller fragments.

But in present days we have much lower error rates and higher speed of communication, this opens a possibility to increase the value of MTU. By increasing value of MTU we will result in less protocol overhead and reduce CPU utilization mostly due to interrupt reduction.

This way some non-standard frames started to emerge:

  • Giant or Jumbo frames - frames that are bigger than standard (IEEE) Ethernet MTU
  • Baby Giant or Baby Jumbo frames - frames that are just slightly bigger that standard (IEEE) Ethernet MTU

It is common now for Ethernet interfaces to support physical MTU above standard, but this can not be taken for granted. Abilities of other network equipment must be taken into account as well - for example, if 2 routers with Ethernet interfaces supporting physical MTU 1526 are connected through Ethernet switch, in order to successfully implement some application that will produce this big Ethernet frames, switch must also support forwarding such frames.

MTU on RouterOS

Different types of MTU


Mikrotik RouterOS recognizes several types of MTU:

  • IP/Layer-3/L3 MTU
  • MPLS/Layer-2.5/L2.5 MTU
  • MAC/Layer-2/L2 MTU
  • Full frame MTU



Full frame MTU

Full frame MTU indicates the actual size of the frame that are sent by particular interface. Frame Checksum is not included as it is removed by Ethernet driver as soon as frame reach its destination.

MAC/Layer-2/L2 MTU

L2MTU indicates the maximum size of the frame without MAC header that can be sent by this interface.

Starting from the RouterOS v3.25 L2MTU values can be seen in "/interface" menu. L2MTU support is added for all Routerboard related Ethernet interfaces, VLANs, Bridge, VPLS and wireless interfaces. Some of them support configuration of L2MTU value. All other Ethernet interfaces might indicate L2MTU only if the chip set is the same as Routerboard Ethernets.

This will allow users to check if desired setup is possible. Users will be able to utilize additional bytes for VLAN and MPLS tags, or simple increase of interface MTU to get rid of the some unnecessary fragmentation.

This table shows max-l2mtu supported by Mikrotik RouterBoards (Starting from the RouterOS v5.3 also available in "/interface print" menu as value of read-only "max-l2mtu" option):

Model name MTU description
RB Groove series ether1:2028
RB QRT series ether1:4076
RB Metal series ether1:2028
RB SXT series, RB LHG, RB LDF, PL6411-2nD, PL7411-2nD, wAP R-2nD, RB912R-2nD-LTm (LtAP mini) ether1:2028
RB SXT Lite series ether1:2028
RB SXT G series, RB DynaDish, wAP ac ether1:4076
RB OmniTik series ether1:4076; ether2-ether5:2028
RB OmniTik ac series ether1-ether5:4074
RB mAP, RB mAP lite, RB cAP, RB wAP ether1-ether2:2028
RB750 ether1:4076; ether2-ether5:2028
RB750r2, RB750P-PBr2, RB750UPr2 ether1-ether5:2028
RB750UP ether1:4076; ether2-ether5:2028
RB751U-2HnD ether1:4076; ether2-ether5:2028
RB951-2n ether1:4076; ether2-ether5:2028
RB941-2nD, RB951Ui/RB952Ui series ether1-ether5:2028
RB750GL, RB750Gr2 ether1-ether5:4074
RB750Gr3 ether1-ether5:2026
RB751G-2HnD ether1-ether5:4074
RB951G-2HnD ether1-ether5:4074
RB962UiGS, RB960PGS ether1-ether5:4074; sfp1:4076
RB1100Hx2 ether1-ether10:9498; ether11:9500; ether12-ether13:9116
RB1100AHx2 ether1-ether10:9498; ether11:9500; ether12-ether13:9116
RB4011iGS+ series ether1-ether10:9578; sfp-sfpplus1:9982
CCR1009 series ether1-ether4:10224; ether5-ether8:10226; sfp1:10226; sfp-sfpplus1:10226
CCR1016 series ether1-ether12:10226; sfp1-sfp12:10226; sfp-sfpplus1:10226
CCR1036 series ether1-ether12:10226; sfp1-sfp4:10226; sfp-sfpplus1-sfp-sfpplus2:10226
CCR1072 series ether1:9116; sfp-sfpplus1-sfp-sfpplus8:10226
CRS109-8G-1S ether1-ether8:4064; sfp1:4064
CRS125-24G-1S ether1-ether24:4064; sfp1:4064
CRS112-8G-4S ether1-ether8:9204; sfp9-sfp12:9204
CRS106-1C-5S sfp1-sfp5:9204; combo1:9204
CRS210-8G-2S+ ether1-ether8:9204; sfp-sfpplus1:9204; sfpplus2:9204
CRS212-1G-10S-1S+ ether1:9204; sfp1-sfp10:9204; sfpplus1:9204
CRS226-24G-2S+ ether1-ether24:9204; sfp-sfpplus1:9204; sfpplus2:9204
CRS326-24G-2S+, CSS326-24G-2S+ ether1-ether24:10218; sfp-sfpplus1:10218; sfpplus2:10218
CRS317-1G-16S+ ether1:10218; sfp-sfpplus1-sfp-sfpplus16:10218
CRS328-24P-4S+ ether1-ether24:10218; sfp-sfpplus1-sfp-sfpplus4:10218
CRS328-4C-20S-4S+ combo1-combo4:10218; sfp1-sfp20:10218; sfp-sfpplus1-sfp-sfpplus4:10218
CRS305-1G-4S+ ether1:10218; sfp-sfpplus1-sfp-sfpplus4:10218
CRS309-1G-8S+ ether1:10218; sfp-sfpplus1-sfp-sfpplus8:10218
CRS312-4C+8XG combo1-combo4:10218; ether1-ether8:10218; ether9:2028
CRS326-24S+2Q+ sfp-sfpplus1-sfp-sfpplus24:10218; qsfpplus1-1-qsfpplus2-4:10218; ether1:2028
D52G-5HacD2HnD (hAP ac²) ether1-ether5:9124
cAP ac ether1-ether2:9124
wAP60G ether1:9124
Cube Lite60 ether1:2028
RB260GS series, CSS106-5G-1S, CSS106-1G-4P-1S ether1-ether5:9198; sfp1:9198
RB FTC ether1:4046; sfp1:4046
RBM33G ether1-ether3:2026
RBM11G ether1:2026
RB760iGS ether1-ether5:2026; sfp1:2026
RB411 series ether1:1526
RB433 series ether1:1526; ether2-ether3:1522
RB450 ether1:1526; ether2-ether5:1522
RB450Gx4 ether1-ether5:9214
RB493 series ether1:1526; ether2-ether9:1522
RB411GL ether1:1520
RB433GL ether1-ether3:1520
RB435G ether1-ether3:1520
RB450G ether1-ether5:1520
RB493G ether1-ether9:1520
RB711 series ether1:2028
RB711G series ether1:4076
RB800 ether1-ether2:9500; ether3:9116
RB850Gx2 ether1-ether5:1580
RB911G ether1:4076
RB912UAG ether1:4076
RB921UAGS, RB922UAGS ether1:4076; sfp1:4076
RB953GS ether1-ether2:4074; sfp1:4074; sfp2:4076
RB2011 series ether1-ether5:4074; ether6-ether10:2028; sfp1:4074
RB3011 series ether1-ether5:8156; ether6-ether10:8156; sfp1:8158
RB44Ge ether1-ether4:9116

Old Products

RouterBoard MTU description
RB600 series ether1-ether3:9500
RB1000 ether1-ether4:9500
RB1100 ether1-ether10:9498; ether11-ether13:9116
RB1100AH ether1-ether10:9498; ether11:9500, ether12-ether13:9116
RB1200 ether1-ether5:4078, ether6-ether8:4080, ether9-ether10:9116
RB750 (old revision) ether1:1526; ether2-ether5:1522
RB750G ether1-ether5:1524
RB333 ether1-ether3:1632
RB1xx ether1-ether5:1518; ether6-ether9:1514
RB532, CrossRoads ether1-ether3:1600
RB44G ether1-ether4:7200
RB44GV ether1-ether4:9000
RB250GS ether1-ether5:9198


All wireless interfaces in RouterOS (including Nstreme2) support 2290 byte L2MTU.

Icon-warn.png

Warning: L2MTU configuration changes evoke all interface reload (link down - link up) due to necessary internal processes.
It is recommended to configure L2MTU with caution by keeping in mind that it can cause short interruption with connected devices.


MPLS/Layer-2.5/L2.5 MTU

Configured in "/mpls interface" menu, specifies maximal size of packet, including MPLS labels, that is allowed to send out by the particular interface (default is 1508).

Make sure that MPLS MTU is smaller or equal to L2MTU

MPLS MTU affects packets depending on what action MPLS router is performing. It is strongly recommended that MPLS MTU is configured to the same value on all routers forming MPLS cloud because of effects MPLS MTU has on MPLS switched packets. This requirement means that all interfaces participating in MPLS cloud must be configured to the smallest MPLS MTU values among participating interfaces, therefore care must be taken to properly select hardware to be used.

MPLS Switching

If packet with labels included is bigger than MPLS MTU, MPLS tries to guess protocol that is carried inside MPLS frame.

If this is IP packet, MPLS produces ICMP Need Fragment error. This behavior mimics IP protocol behavior. Note that this ICMP error is not routed back to originator of packet but is switched towards end of LSP, so that egress router can route it back.

If this is not IP packet, MPLS simply drops it, because it does not know how to interpret the contents of packet. This feature is very important in situations where MPLS applications such as VPLS are used (where frames that are MPLS tagged are not IP packets, but e.g. encapsulated Ethernet frames as in case of VPLS) - if somewhere along the LSP MPLS MTU will be less than packet size prepared by ingress router, frames will simply get dropped.

IP ingress

When router first introduces label (or labels) on IP packet, and resulting packet size including MPLS labels exceeds MPLS MTU, router behaves as if interface MTU was exceeded - either fragments packet in fragments that does not exceed MPLS MTU when labels are attached (if IP Dont Fragment is not set), or generates ICMP Need Fragmentation error that is sent back to originator.

VPLS ingress

When router encapsulates Ethernet frame for forwarding over VPLS pseudowire, it checks if packet size with VPLS Control Word (4 bytes) and any necessary labels (usually 2 labels - 8 bytes), exceeds MPLS MTU of outgoing interface. If it does, VPLS fragments packet so that it honours MPLS MTU of outgoing interface. Packet is defragmented at egress point of VPLS pseudowire.

IP/Layer-3/L3 MTU

Configured as interface MTU setting (/interface <type> <name> set mtu=X). Specifies how big IP packets router is allowed to send out the particular interface.

If router receives IP packet of size 1500, but MTU for outgoing interface is set to 1400, router will either fragment the packet (if "Don't Fragment" bit is not set in IP header) or drop the packet and send ICMP "Need Fragmentation" error back to originator (this is essential for Path MTU Discovery to work).

Sometimes it can be bad idea to change IP MTU from its default 1500 bytes on router interfaces if complete path end-to-end is not in administrators control. Although IP fragmentation and end-to-end Path MTU Discovery is intended to handle this situation, if ICMP Need Fragmentation errors are filtered somewhere along the path, Path MTU Discovery will not work.

There are several features in MikroTik RouterOS that can benefit from possibility to exceed standard MTU

Simple Examples

In these examples we will take a look at frames entering and leaving router via Ethernet interfaces.

Simple Routing

The image shows the packet MTU size for simple routing, packets size is not modified.


MTUSimpleRouting.png


Routing with VLAN Encap

Each VLAN tag is 4 bytes long, VLAN tag is added by router. L2-MTU is increased by 4 bytes.


MTUVLANENCAP.png


Simple MPLS with tags

When MPLS is used as plain replacement for IP routing, only one label is attached to every packet, therefore packet size increases by 4 bytes, we have the situation with two MPLS labels. In order to be able to forward standard size (1500 bytes) IP packet without fragmentation, MPLS MTU must be set to at least 1508 for two MPLS labels.


MTUMPLS2Tags.png


VPLS Tunnel

Two MPLS labels are present, when remote endpoint is not directly attached. One MPLS label is used to get to remote endpoint, second label is used to identify VPLS tunnel.


MTUVPLS.png


L2MTU advanced example

In this example we will take a closer look at required L2MTU of all Ethernet like interfaces including Bridge, VLAN, VPLS interfaces.

In this setup we will have 3 routers:

  • Q-in-Q router - this router will receive standard 1500 byte Ethernet frame and will add two VLAN tags to the packet. Then packet will be sent out via Ethernet network to the second router
  • VPLS router - this router will remove outer VLAN tag and will bridge packet with the remaining VLAN tag with VPLS tunnel. VPLS tunnel will take packet through the MPLS network to the third router.
  • MPLS Edge router - will remove VPLS and VLAN tags and bridge packet to the client Ethernet network.


L2MTU example.png


[ Top | Back to Content ]