Manual:EBGP as PE-CE routing protocol

Applies to RouterOS: v4

• Packages required: routing, mpls
• Software versions: 4.3+

Setup

In this setup we describe the use of EBGP as Provider Edge - Customer Edge (PE-CE) routing protocol.

Router A and Router F both belong to the same customer's VPN, but to different sites.

Router A is multihomed - is has connections to two PEs, router B and router C.

Routers B, C, and E are PE routers.

Router D is provider (P) router and functions as BGP route reflector.

All provider's routers belong to AS 100; all customer routers belong to private AS 65000.

Description

There are several tricky aspects about this setup.

First, it is not possible to use BGP built-in mechanism of routing loop prevention, that checks BGP AS path for presence of local AS path numbers and discards all routes that match. We want to distribute routes from A to F, and vice versa, but they belong to the same BGP AS. (One solution is to use different private AS numbers there, but that's not always possible or desirable.)

• One way to do work around this BGP AS path loop check is to configure BGP as-override option at exit point from provider's network.
• Another way is to configure remove-private-as at providers network entry point (it will work only if customer's AS numbers are private, of course!)
• Yet another way is to configure allow-as-in=x on customers edge router. "x" is the number of times local as number can be present in AS path.

In this configuration we use the as-override option on router E (to make router F accept routes from A), and allow-as-in option on router A, to make it accept routes from F.

Router A:

routing bgp peer add remote-address=10.1.1.2 remote-as=100 allow-as-in=1;
routing bgp peer add remote-address=10.1.1.6 remote-as=100 allow-as-in=1;

Router E:

routing bgp peer add instance=ebgp remote-address=10.3.3.2 remote-as=65000 as-override=yes;

The second tricky aspect is that since CE1 is multihomed (i.e. has links to multiple PEs) and BGP AS path loop prevention mechanism is disabled on router A because 'allow-as-in' option configured, the routes that A advertises to one PE router may be received back from the second PE. Installing those route in VRF table can also lead to suboptimal routing and even to BGP convergence failure. To avoid that, BGP Site of Origin (SOO) extended communities can be used. In this configuration we configure routing filter on PE routers that sets BGP SOO extended communities to routes received from CE router, and another filter, that filters out VPNv4 routes received from IBGP by the same SOO extended community attribute.

Routers B, C:

routing filter add chain=ibgp-in site-of-origin=1:100 action=discard;
routing filter add chain=ebgp-in set-site-of-origin=1:100;

We also use different BGP instances on PE routers: one for PE-CE (i.e. EBGP) peers and one for provider's network internal BGP peers.

Configuration

Router A:

 ip address add address=10.1.1.1/30 interface=A_B;
 ip address add address=10.1.1.5/30 interface=A_C;
 interface bridge add name=somenet;
 ip address add address=10.10.10.1/24 interface=somenet;
 routing bgp instance set default as=65000 redistribute-connected=yes;
 routing bgp peer add remote-address=10.1.1.2 remote-as=100 allow-as-in=1;
 routing bgp peer add remote-address=10.1.1.6 remote-as=100 allow-as-in=1;

Router B:

ip address add address=10.1.1.2/30 interface=B_A;
ip address add address=10.2.2.1/30 interface=B_D;
interface bridge add name=lobridge;
ip address add address=10.9.9.2/32 interface=lobridge;
ip route add dst-address=10.9.9.3 gateway=10.2.2.2;
ip route add dst-address=10.9.9.4 gateway=10.2.2.2;
ip route add dst-address=10.9.9.5 gateway=10.2.2.2;
ip route vrf add routing-mark=vrf1 interfaces=B_A route-distinguisher=1:1 \
 import-route-targets=1:1 export-route-targets=1:1;
mpls ldp set enabled=yes transport-address=10.9.9.2;
mpls ldp interface add interface=B_D hello-interval=3;
routing bgp instance set default as=100;
routing bgp instance add name=ebgp router-id=0.0.0.2 as=100 routing-table=vrf1;
routing bgp instance vrf add instance=default routing-mark=vrf1 redistribute-connected=yes \
 redistribute-other-bgp=yes;
routing bgp peer add address-families=vpnv4 remote-address=10.9.9.4 remote-as=100 \
 in-filter=ibgp-in out-filter=ibgp-out update-source=10.9.9.2;
routing bgp peer add instance=ebgp remote-address=10.1.1.1 remote-as=65000 \
  in-filter=ebgp-in out-filter=ebgp-out;
routing filter add chain=ebgp-out site-of-origin=1:100 action=discard;
routing filter add chain=ebgp-in set-site-of-origin=1:100;

Router C:

ip address add address=10.1.1.6/30 interface=C_A;
ip address add address=10.2.2.5/30 interface=C_D;
interface bridge add name=lobridge;
ip address add address=10.9.9.3/32 interface=lobridge;
ip route add dst-address=10.9.9.2 gateway=10.2.2.6;
ip route add dst-address=10.9.9.4 gateway=10.2.2.6;
ip route add dst-address=10.9.9.5 gateway=10.2.2.6;
ip route vrf add routing-mark=vrf1 interfaces=C_A route-distinguisher=1:1 \
 import-route-targets=1:1 export-route-targets=1:1;
mpls ldp set enabled=yes transport-address=10.9.9.3;
mpls ldp interface add interface=C_D hello-interval=3;
routing bgp instance set default as=100;
routing bgp instance add name=ebgp router-id=0.0.0.3 as=100 routing-table=vrf1;
routing bgp instance vrf add instance=default routing-mark=vrf1 \
 redistribute-connected=yes redistribute-other-bgp=yes;
routing bgp peer add address-families=vpnv4 remote-address=10.9.9.4 remote-as=100 \
 in-filter=ibgp-in update-source=10.9.9.3;
routing bgp peer add instance=ebgp remote-address=10.1.1.5 remote-as=65000 \
 in-filter=ebgp-in out-filter=ebgp-out;
routing filter add chain=ibgp-in site-of-origin=1:100 action=discard;
routing filter add chain=ebgp-in set-site-of-origin=1:100;

Router D:

ip address add address=10.2.2.2/30 interface=D_B;
ip address add address=10.2.2.6/30 interface=D_C;
ip address add address=10.2.2.9/30 interface=D_E;
interface bridge add name=lobridge;
ip address add address=10.9.9.4/32 interface=lobridge;
ip route add dst-address=10.9.9.2 gateway=10.2.2.1;
ip route add dst-address=10.9.9.3 gateway=10.2.2.5;
ip route add dst-address=10.9.9.5 gateway=10.2.2.10;
mpls ldp set enabled=yes transport-address=10.9.9.4;
mpls ldp interface add interface=D_B hello-interval=3;
mpls ldp interface add interface=D_C hello-interval=3;
mpls ldp interface add interface=D_E hello-interval=3;
routing bgp instance set default as=100;
routing bgp peer add address-families=vpnv4 remote-address=10.9.9.2 remote-as=100 \
 update-source=10.9.9.4 route-reflect=yes;
routing bgp peer add address-families=vpnv4 remote-address=10.9.9.3 remote-as=100 \
 update-source=10.9.9.4 route-reflect=yes;
routing bgp peer add address-families=vpnv4 remote-address=10.9.9.5 remote-as=100 \
 update-source=10.9.9.4 route-reflect=yes;

Router E:

 ip address add address=10.3.3.1/30 interface=E_F;
 ip address add address=10.2.2.10/30 interface=E_D;
 interface bridge add name=lobridge;
 ip address add address=10.9.9.5/32 interface=lobridge;
 ip route add dst-address=10.9.9.2 gateway=10.2.2.9;
 ip route add dst-address=10.9.9.3 gateway=10.2.2.9;
 ip route add dst-address=10.9.9.4 gateway=10.2.2.9;
 ip route vrf add routing-mark=vrf1 interfaces=E_F route-distinguisher=1:1 \
  import-route-targets=1:1 export-route-targets=1:1;
 mpls ldp set enabled=yes transport-address=10.9.9.5;
 mpls ldp interface add interface=E_D hello-interval=3;
 routing bgp instance set default as=100;
 routing bgp instance add name=ebgp router-id=0.0.0.5 as=100 routing-table=vrf1;
 routing bgp instance vrf add instance=default routing-mark=vrf1 redistribute-connected=yes \
  redistribute-other-bgp=yes;
 routing bgp peer add address-families=vpnv4 remote-address=10.9.9.4 remote-as=100 \
  update-source=10.9.9.5;
 routing bgp peer add instance=ebgp remote-address=10.3.3.2 remote-as=65000 as-override=yes;

Router F:

ip address add address=10.3.3.2/30 interface=F_E;
interface bridge add name=somenet;
ip address add address=10.20.20.1/24 interface=somenet;
routing bgp instance set default as=65000 redistribute-connected=yes;
routing bgp peer add remote-address=10.3.3.1 remote-as=100;

Results

Routes on CE1 router A:

 [admin@A] > ip route print detail
 Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, 
 o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit

1 ADC dst-address=10.1.1.4/30 pref-src=10.1.1.5 gateway=A_C gateway-status=A_C reachable 
 distance=0 scope=10

2 ADb dst-address=10.3.3.0/30 gateway=10.1.1.2 gateway-status=10.1.1.2 reachable A_B
 distance=20 scope=40 target-scope=10 bgp-as-path=100 bgp-origin=incomplete 
 bgp-ext-communities=RT:1:1 received-from=peer1

3  Db dst-address=10.3.3.0/30 gateway=10.1.1.6 gateway-status=10.1.1.6 reachable A_C
 distance=20 scope=40 target-scope=10 bgp-as-path=100 bgp-origin=incomplete 
 bgp-ext-communities=RT:1:1 received-from=peer2

4 ADC dst-address=10.10.10.1/30 pref-src=10.1.1.1 gateway=somenet 
 gateway-status=somenet reachable distance=0 scope=10

5 ADb dst-address=10.20.20.0/24 gateway=10.1.1.2 gateway-status=10.1.1.2 reachable A_B
 distance=20 scope=40 target-scope=10 bgp-as-path=100,65000 bgp-origin=incomplete 
 bgp-ext-communities=RT:1:1 received-from=peer1

6  Db dst-address=10.20.20.0/24 gateway=10.1.1.6 gateway-status=10.1.1.6 reachable A_C
 distance=20 scope=40 target-scope=10 bgp-as-path=100,65000 bgp-origin=incomplete 
 bgp-ext-communities=RT:1:1 received-from=peer2

Routes on CE2 router F:

 [admin@F] > ip route print detail
 Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, 
 o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit

0 ADb dst-address=10.1.1.0/30 gateway=10.3.3.1 gateway-status=10.3.3.1 reachable F_E
 distance=20 scope=40 target-scope=10 bgp-as-path=100 bgp-origin=incomplete 
 bgp-ext-communities=RT:1:1 received-from=peer1

1 ADb dst-address=10.1.1.4/30 gateway=10.3.3.1 gateway-status=10.3.3.1 reachable F_E
 distance=20 scope=40 target-scope=10 bgp-as-path=100 bgp-origin=incomplete 
 bgp-ext-communities=RT:1:1 received-from=peer1

2 ADC dst-address=10.3.3.0/30 pref-src=10.3.3.2 gateway=F_E gateway-status=F_E reachable 
 distance=0 scope=10

3 ADb dst-address=10.10.10.0/24 gateway=10.3.3.1 gateway-status=10.3.3.1 reachable F_E
 distance=20 scope=40 target-scope=10 bgp-as-path=100,100 bgp-origin=incomplete
 bgp-ext-communities=RT:1:1,SOO:1:100 received-from=peer1

4 ADC dst-address=10.20.20.0/30 pref-src=10.20.20.1 gateway=somenet 
 gateway-status=somenet reachable distance=0 scope=10

Routes on PE1 router B:

 [admin@B] > ip route print detail
 Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, 
 o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit

0 ADC dst-address=10.1.1.0/30 pref-src=10.1.1.2 gateway=B_A gateway-status=B_A reachable 
 distance=0 scope=10 routing-mark=vrf1

1  Db dst-address=10.1.1.0/30 gateway=10.1.1.1 gateway-status=10.1.1.1 on vrf1 reachable A_B
 distance=20 scope=40 target-scope=10 routing-mark=vrf1 bgp-as-path=65000 
 bgp-origin=incomplete bgp-ext-communities=SOO:1:100 received-from=peer2

2 ADb dst-address=10.1.1.4/30 =gateway=10.1.1.1 gateway-status=10.1.1.1 on vrf1 reachable B_A
 distance=20 scope=40 target-scope=10 routing-mark=vrf1 bgp-as-path=65000 
 bgp-origin=incomplete bgp-ext-communities=SOO:1:100 received-from=peer2

3  Db dst-address=10.1.1.4/30 gateway=10.9.9.3 gateway-status=10.9.9.3 recursive via 10.2.2.2 B_D
 distance=20 scope=40 target-scope=30 routing-mark=vrf1 bgp-local-pref=100 bgp-origin=incomplete
 bgp-ext-communities=RT:1:1

4 ADb dst-address=10.3.3.0/30 gateway=10.9.9.5 gateway-status=10.9.9.5 recursive via 10.2.2.2 B_D
 distance=20 scope=40 target-scope=30 routing-mark=vrf1 bgp-local-pref=100 bgp-origin=incomplete
 bgp-ext-communities=RT:1:1

5 ADb dst-address=10.10.10.0/24 gateway=10.1.1.1 gateway-status=10.1.1.1 on vrf1 reachable B_A
 distance=20 scope=40 target-scope=10 routing-mark=vrf1 bgp-as-path=65000 bgp-origin=incomplete
 bgp-ext-communities=SOO:1:100 received-from=peer2

6 ADb dst-address=10.20.20.0/24 gateway=10.9.9.5 gateway-status=10.9.9.5 recursive via 10.2.2.2 B_D
 distance=20 scope=40 target-scope=30 routing-mark=vrf1 bgp-as-path=65000 bgp-local-pref=100
 bgp-origin=incomplete bgp-ext-communities=RT:1:1

7 ADC dst-address=10.2.2.0/30 pref-src=10.2.2.1 gateway=B_D gateway-status=B_D reachable
 distance=0 scope=10

8 ADC dst-address=10.9.9.2/32 pref-src=10.9.9.2 gateway=lobridge gateway-status=lobridge reachable
 distance=0 scope=10

9 A S dst-address=10.9.9.3/32 gateway=10.2.2.2 gateway-status=10.2.2.2 reachable B_D
 distance=1 scope=30 target-scope=10

10 A S dst-address=10.9.9.4/32 gateway=10.2.2.2 gateway-status=10.2.2.2 reachable B_D
 distance=1 scope=30 target-scope=10

11 A S dst-address=10.9.9.5/32 gateway=10.2.2.2 gateway-status=10.2.2.2 reachable B_D
 distance=1 scope=30 target-scope=10

See also

Corresponding test script