Manual:Basic MPLS setup example
Applies to RouterOS: v3.17 +
Introduction
MPLS or Multiprotocol Label Switching is used widely for it's performance and traffic engineering possibilities. While MPLS can seem complex, this example will show how to get a very basic MPLS setup up and running. In this example we will be focusing ONLY on making traffic between CE1 and CE2 routable and route it using MPLS.
Configuration
Loopback interfaces
For stability reasons it is recommended to create a loopback interface on each router that is involved either in dynamic routing or MPLS. If you assign an IP address on a interface that is not a loopback interface, then the IP address will be inactive along with the interface, this will cause problems for routing protocols even if there are multiple paths to a single router. It is highly recommended that you use loopback interfaces in MPLS setups.
- Use the following commands on PE1, P1, P2, P3 and PE2 to create loopback interfaces:
/interface bridge add name=loopback protocol-mode=none
| Address | Device |
|---|---|
| 10.0.0.1 | PE1 |
| 10.0.0.2 | P1 |
| 10.0.0.3 | P2 |
| 10.0.0.4 | P3 |
| 10.0.0.5 | PE2 |
- Assign proper IP addresses on the loopback interface for each device (replace the X), use the following commands on PE1, P1, P2, P3 and PE2:
/ip address add address=10.0.0.X/32 interface=loopback
Routing
Before MPLS can be enabled in a network, routing must be set up. In this example we don't require traffic from any P or PE to be routable, we only require that CE1 and CE2 is able to route traffic to each other. When using dynamic routing like OSPF or BGP, these routes can be advertised across the whole network. For this example we are going to use static routing and to make traffic from/to CE1 to/from CE2 routable you only a few static routes and IP addresses on each interface that is connected to a router. We are going to need routes that point to 192.168.1.0/24 and 192.168.6.0/24 networks for routers that are not connected directly to respective network.
In this example it is required not only for the traffic between CE1 and CE2 to be routable, but also the traffic between each MPLS enabled router to be routable using the loopback IP address. Routes for loopback IP addresses can also be advertised using dynamic routing protocols, but to make this setup as simple and straightforward as possible, we are going to use static routes for loopback IP addresses as well. We are going to need to create routes for reach loopback IP address in the whole network. See static routing example here.
- Use the following commands on CE1:
/ip address add address=192.168.1.1/24 interface=ether1 network=192.168.1.0 /ip route add gateway=192.168.1.2
- Use the following commands on PE1:
/ip address add address=192.168.1.2/24 interface=ether1 add address=192.168.2.1/24 interface=ether2 /ip route add dst-address=10.0.0.2/32 gateway=192.168.2.2 add dst-address=10.0.0.3/32 gateway=192.168.2.2 add dst-address=10.0.0.4/32 gateway=192.168.2.2 add dst-address=10.0.0.5/32 gateway=192.168.2.2 add dst-address=192.168.6.0/24 gateway=192.168.2.2
- Use the following commands on P1:
/ip address add address=192.168.2.2/24 interface=ether1 add address=192.168.3.1/24 interface=ether2 /ip route add dst-address=10.0.0.1/32 gateway=192.168.2.1 add dst-address=10.0.0.3/32 gateway=192.168.3.2 add dst-address=10.0.0.4/32 gateway=192.168.3.2 add dst-address=10.0.0.5/32 gateway=192.168.3.2 add dst-address=192.168.1.0/24 gateway=192.168.2.1 add dst-address=192.168.6.0/24 gateway=192.168.3.2
- Use the following commands on P2:
/ip address add address=192.168.3.2/24 interface=ether1 add address=192.168.4.1/24 interface=ether2 /ip route add dst-address=10.0.0.1/32 gateway=192.168.3.1 add dst-address=10.0.0.2/32 gateway=192.168.3.1 add dst-address=10.0.0.4/32 gateway=192.168.4.2 add dst-address=10.0.0.5/32 gateway=192.168.4.2 add dst-address=192.168.1.0/24 gateway=192.168.3.1 add dst-address=192.168.6.0/24 gateway=192.168.4.2
- Use the following commands on P3:
/ip address add address=192.168.5.1/24 interface=ether1 add address=192.168.4.2/24 interface=ether2 /ip route add dst-address=10.0.0.1/32 gateway=192.168.4.1 add dst-address=10.0.0.2/32 gateway=192.168.4.1 add dst-address=10.0.0.3/32 gateway=192.168.4.1 add dst-address=10.0.0.5/32 gateway=192.168.5.2 add dst-address=192.168.1.0/24 gateway=192.168.4.1 add dst-address=192.168.6.0/24 gateway=192.168.5.2
- Use the following commands on PE2:
/ip address add address=192.168.6.1/24 interface=ether1 add address=192.168.5.2/24 interface=ether2 /ip route add dst-address=10.0.0.1/32 gateway=192.168.5.1 add dst-address=10.0.0.2/32 gateway=192.168.5.1 add dst-address=10.0.0.3/32 gateway=192.168.5.1 add dst-address=10.0.0.4/32 gateway=192.168.5.1 add dst-address=192.168.1.0/24 gateway=192.168.5.1
- Use the following commands on CE2:
/ip address add address=192.168.6.2/24 interface=ether1 /ip route add gateway=192.168.6.1
- Test if CE1 can reach CE2 and check if the packet is routed through all routers in the path, use the following commands on CE1:
[admin@CE1] > /ping 192.168.6.2
SEQ HOST SIZE TTL TIME STATUS
0 192.168.6.2 56 59 0ms
1 192.168.6.2 56 59 0ms
2 192.168.6.2 56 59 0ms
sent=3 received=3 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
[admin@CE1] > /tool traceroute 192.168.6.2
# ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV STATUS
1 192.168.1.2 0% 6 0.2ms 0.2 0.2 0.3 0
2 192.168.2.2 0% 6 0.2ms 0.2 0.2 0.2 0
3 192.168.3.2 0% 6 0.2ms 0.2 0.1 0.2 0
4 192.168.4.2 0% 6 0.2ms 0.2 0.2 0.3 0
5 192.168.5.2 0% 6 0.2ms 0.2 0.2 0.3 0
6 192.168.6.2 0% 6 0.2ms 0.2 0.2 0.3 0
MPLS
When routing between CE1 and CE2 is working properly, you can enable MPLS. You only need to change the transport-address for each MPLS router, which is the same as the loopback IP address, you also need to specify which interfaces are involved into MPLS.
- Use the following commands on PE1:
/mpls ldp set enabled=yes transport-address=10.0.0.1 /mpls ldp interface add interface=ether2
- Use the following commands on P1:
/mpls ldp set enabled=yes transport-address=10.0.0.2 /mpls ldp interface add interface=ether1 add interface=ether2
- Use the following commands on P2:
/mpls ldp set enabled=yes transport-address=10.0.0.3 /mpls ldp interface add interface=ether1 add interface=ether2
- Use the following commands on P3:
/mpls ldp set enabled=yes transport-address=10.0.0.4 /mpls ldp interface add interface=ether1 add interface=ether2
- Use the following commands on PE2:
/mpls ldp set enabled=yes transport-address=10.0.0.5 /mpls ldp interface add interface=ether2
- Check if all labels have been distributed, there should be 2 labels for both end networks and labels for all loopback IP addresses (except for router's own loopback IP address):
[admin@P2] /mpls> forwarding-table print Flags: H - hw-offload, L - ldp, V - vpls, T - traffic-eng # IN-LABEL OUT-LABELS DESTINATION INTERFACE NEXTHOP 0 expl-null 1 L 25 25 192.168.1.0/24 ether1 192.168.3.1 2 L 26 27 10.0.0.1/32 ether1 192.168.3.1 3 L 27 10.0.0.2/32 ether1 192.168.3.1 4 L 28 27 10.0.0.5/32 ether2 192.168.4.2 5 L 29 25 192.168.6.0/24 ether2 192.168.4.2 6 L 30 10.0.0.4/32 ether2 192.168.4.2
- Test if CE1 can reach CE2 through the MPLS network, use the following commands on CE1:
[admin@CE1] > /ping 192.168.6.2
SEQ HOST SIZE TTL TIME STATUS
0 192.168.6.2 56 59 0ms
1 192.168.6.2 56 59 0ms
2 192.168.6.2 56 59 0ms
sent=3 received=3 packet-loss=0% min-rtt=0ms avg-rtt=0ms max-rtt=0ms
[admin@CE1] > /tool traceroute 192.168.6.2
# ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV STATUS
1 192.168.1.2 0% 3 0.1ms 0.1 0.1 0.2 0
2 192.168.2.2 0% 3 0.1ms 0.1 0.1 0.2 0 <MPLS:L=28,E=0>
3 100% 3 timeout
4 192.168.4.2 0% 2 0.3ms 0.3 0.3 0.3 0 <MPLS:L=25,E=0>
5 192.168.5.2 0% 2 0.1ms 0.1 0.1 0.1 0
6 192.168.6.2 0% 2 0.2ms 0.2 0.2 0.2 0
Note: It is expected that the first P router in path will timeout when using traceroute, you can hide the MPLS network by setting propagate-ttl to no
MPLS hardware offloading
Some devices are capable of offloading certain MPLS functions. In this example the router P2 can offload label switching to the hardware on CRS317-1G-16S+, this means that the CPU is not going to be used for label switching and the built-in switch chip can do this at wire-speed. This setup can be simplified to a single P router and the CRS317-1G-16S+ still will be capable of using hardware offloading, but it requires to use explicit null across the whole MPLS network. You can check if label switching is offloaded to the hardware by checking the forwarding table:
[admin@P2] /mpls> forwarding-table print Flags: H - hw-offload, L - ldp, V - vpls, T - traffic-eng # IN-LABEL OUT-LABELS DESTINATION INTERFACE NEXTHOP 0 expl-null 1 HL 25 25 192.168.1.0/24 sfp-sfpplus1 192.168.3.1 2 L 26 27 10.0.0.1/32 sfp-sfpplus1 192.168.3.1 3 L 27 10.0.0.2/32 sfp-sfpplus1 192.168.3.1 4 L 28 27 10.0.0.5/32 sfp-sfpplus2 192.168.4.2 5 HL 29 25 192.168.6.0/24 sfp-sfpplus2 192.168.4.2 6 L 30 10.0.0.4/32 sfp-sfpplus2 192.168.4.2
Note: Currently only CRS317-1G-16S+ and CRS309-1G-8S+ using RouterOS v6.41 and newer are capable of hardware offloading certain MPLS functions. CRS317-1G-16S+ and CRS309-1G-8S+ built-in switch chip is not capable of popping MPLS labels from packets, in a PE-P-PE setup you either have to use explicit null or disable TTL propagation in MPLS network to achieve hardware offloading.