Manual:Layer-3 MPLS VPN example
This is a kind of "putting it all together" setup.
Technologies used:
- LDP for MPLS label distribution
- BGP for VPNv4 route distribution
- OSPF as CE - PE routing protocol
Software:
- PE and P routers has RouterOS 3.17 with routing-test and mpls-test packages.
- CE routers has RouterOS 3.17 with routing-test package. (routing package and older versions can be used here as well.)
IP addressing & routing
Provider's network
On Router B:
/ip address add address=10.1.1.2/24 interface=ether2 /ip address add address=10.2.2.2/24 interface=ether3 # put PE-CE interface in a VRF /ip route vrf add routing-mark=vrf1 interfaces=ether2 \ route-distinguisher=10.1.1.1:111 import-route-targets=10.1.1.1:111 export-route-targets=10.1.1.1:111 # loopback interace /interface bridge add name=lobridge /ip address add address=10.9.9.2/32 interface=lobridge # add routes to loopback addresses # (static routing is used for destioatins inside providers network) /ip route add dst-address=10.9.9.3/32 gateway=10.2.2.3 /ip route add dst-address=10.9.9.4/32 gateway=10.2.2.3
On Router C:
/ip address add address=10.2.2.3/24 interface=ether3 /ip address add address=10.3.3.3/24 interface=ether2 # loopback interace /interface bridge add name=lobridge /ip address add address=10.9.9.3/32 interface=lobridge # add routes to loopback addresses /ip route add dst-address=10.9.9.2/32 gateway=10.2.2.2 /ip route add dst-address=10.9.9.4/32 gateway=10.3.3.4
On Router D:
/ip address add address=10.3.3.4/24 interface=ether2 /ip address add address=10.4.4.4/24 interface=ether3 # put PE-CE interface in a VRF /ip route vrf add routing-mark=vrf1 interfaces=ether3 \ route-distinguisher=10.1.1.1:111 import-route-targets=10.1.1.1:111 export-route-targets=10.1.1.1:111 # loopback interace /interface bridge add name=lobridge /ip address add address=10.9.9.4/32 interface=lobridge # add routes to loopback addresses /ip route add dst-address=10.9.9.2/32 gateway=10.3.3.3 /ip route add dst-address=10.9.9.3/32 gateway=10.3.3.3
Client's sites
On Router A:
/ip address add address=10.1.1.1/24 interface=<ToRouterB>
On Router E:
/ip address add address=10.4.4.5/24 interface=<ToRouterD> /ip address add address=10.7.7.5/24 interface=<ToLocalNetwork>
LDP
On Router B:
/mpls ldp set enabled=yes transport-address=10.9.9.2 /mpls ldp interface add interface=ether3
On Router C:
/mpls ldp set enabled=yes transport-address=10.9.9.3 /mpls ldp interface add interface=ether2 /mpls ldp interface add interface=ether3
On Router D:
/mpls ldp set enabled=yes transport-address=10.9.9.4 /mpls ldp interface add interface=ether2
Setting transport address for LDP is not required, but very recommended. If the address is not set, the router will pick any address at random, which may be an address belonging to VRF, and as such not connectible from internal P routers.
Results
[admin@C] > /mpls ldp neighbor print
Flags: X - disabled, D - dynamic, O - operational, T - sending-targeted-hello, V - vpls
# TRANSPORT LOCAL-TRANSPORT PEER SEN ADDRESSES
0 O 10.9.9.2 10.9.9.3 10.1.1.2:0 no 10.1.1.2
10.2.2.2
10.9.9.2
10.3.0.2
1 10.3.3.4 no
2 O 10.9.9.4 10.9.9.3 10.3.3.4:0 no 10.3.3.4
10.4.4.4
10.9.9.4
10.3.0.4
BGP
On Router B:
/routing bgp instance set default vrf=vrf1 redistribute-connected=yes redistribute-ospf=yes /routing bgp peer add remote-address=10.2.2.3 address-families=vpnv4 remote-as=64550 update-source=lobridge
On Router C:
/routing bgp peer add remote-address=10.2.2.2 remote-as=64550 route-reflect=yes address-families=vpnv4 /routing bgp peer add remote-address=10.3.3.4 remote-as=64550 route-reflect=yes address-families=vpnv4 # client-to-client-reflection is on by default #/routing bgp instance set default client-to-client-reflection=yes
On Router D:
/routing bgp instance set default vrf=vrf1 redistribute-connected=yes redistribute-ospf=yes /routing bgp peer add remote-address=10.3.3.3 address-families=vpnv4 remote-as=64550 update-source=lobridge
Note that route reflection here is used for the sake of an example. A simpler configuration would work as well - one where there is a BGP session between B and D and C is not running BGP at all.
OSPF
On Router A:
/routing ospf network add network=10.1.1.0/24 area=backbone
On Router B:
/routing ospf instance set default routing-table=vrf1 redistribute-bgp=as-type-1 /routing ospf network add network=10.1.1.0/24 area=backbone
On Router D:
/routing ospf instance set default routing-table=vrf1 redistribute-bgp=as-type-1 /routing ospf network add network=10.4.4.0/24 area=backbone
On Router E:
/routing ospf network add network=10.4.4.0/24 area=backbone /routing ospf network add network=10.7.7.0/24 area=backbone
Results
Routing table on CE router A:
[admin@A] > /ip route pr Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit # DST-ADDRESS PREF-SRC GATEWAY DISTANCE 0 A S 0.0.0.0/0 10.3.0.254 reac... 1 1 ADC 10.1.1.0/24 10.1.1.1 ether2 0 2 ADo 10.4.4.0/24 10.1.1.2 reachab... 110 3 ADo 10.7.7.0/24 10.1.1.2 reachab... 110 4 ADC 10.9.9.1/32 10.9.9.1 lobridge 0 5 ADC 10.3.0.0/24 10.3.0.1 ether1 0
Routing table on CE router E:
[admin@E] > /ip route pr Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit # DST-ADDRESS PREF-SRC GATEWAY DISTANCE 0 A S 0.0.0.0/0 10.3.0.254 reac... 1 1 ADo 10.1.1.0/24 10.4.4.4 reachab... 110 2 ADC 10.4.4.0/24 10.4.4.5 ether2 0 3 ADC 10.7.7.0/24 10.7.7.5 ether3 0 4 ADC 10.3.0.0/24 10.3.0.5 ether1 0
Test
On Router A:
Ping from CE1 -> to PE1:
[admin@A] > /ping 10.1.1.2 10.1.1.2 64 byte ping: ttl=64 time=8 ms 10.1.1.2 64 byte ping: ttl=64 time=4 ms 10.1.1.2 64 byte ping: ttl=64 time=5 ms 10.1.1.2 64 byte ping: ttl=64 time=5 ms 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 4/5.5/8 ms
Ping from CE1 -> to CE2:
[admin@A] > /ping 10.4.4.5 10.4.4.5 64 byte ping: ttl=61 time=12 ms 10.4.4.5 64 byte ping: ttl=61 time=5 ms 10.4.4.5 64 byte ping: ttl=61 time=6 ms 10.4.4.5 64 byte ping: ttl=61 time=8 ms 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 5/7.7/12 ms
[admin@A] > /ping 10.7.7.5
10.7.7.5 64 byte ping: ttl=61 time=14 ms 10.7.7.5 64 byte ping: ttl=61 time=4 ms 10.7.7.5 64 byte ping: ttl=61 time=8 ms 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 4/8.6/14 ms
[admin@A] > /tool traceroute 10.7.7.5
ADDRESS STATUS
1 10.1.1.2 3ms 6ms 2ms
2 0.0.0.0 timeout timeout timeout
3 10.3.3.4 4ms 3ms 3ms
4 10.7.7.5 3ms 3ms 3ms
The second hop failure is normal.
To see whole MPLS cloud as one IP hop, configure propagate-ttl=no. This setting should be the same on all provider's routers.
On Routers B,C,D:
/mpls set propagate-ttl=no
[admin@A] > /tool traceroute 10.7.7.5
ADDRESS STATUS
1 10.1.1.2 6ms 3ms 5ms
2 10.3.3.4 5ms 3ms 6ms
3 10.7.7.5 9ms 9ms 6ms
No failures here.
You can also connect from PE to CE, but in this case routing-table must be specified manually.
Ping from PE1 -> to CE1:
[admin@B] > ping 10.1.1.2 routing-table=vrf1 10.1.1.2 64 byte ping: ttl=64 time=9 ms 10.1.1.2 64 byte ping: ttl=64 time=6 ms 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 6/7.5/9 ms
Ping from PE1 -> to PE2 works as well:
[admin@B] > /ping 10.9.9.4 src-address=10.9.9.2 10.9.9.4 64 byte ping: ttl=63 time=17 ms 10.9.9.4 64 byte ping: ttl=63 time=12 ms 10.9.9.4 64 byte ping: ttl=63 time=10 ms 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 10/13.0/17 ms