Manual:BGP based VPLS

Overview

MPLSVPLS page covers general introduction to VPLS service and configuration of LDP based VPLS tunnels. Due to their static nature, LDP based VPLS tunnels have scalability issues that arise when number of VPLSes and sites participating in VPLSes grow. One of the problems is the requirement to maintan full mesh of LDP tunnels between sites forming VPLS. In case number of sites in VPLS is high, adding new site to existing VPLS can become burdensome for network administrator.

BGP based autodiscovery and signaling of VPLS tunnels can help to avoid complexity of configuration at the expense of running BGP protocol between VPLS routers. In general, BGP based VPLS serves two purposes:

• autodiscovery: there is no need to configure each VPLS router with all remote endpoints of VPLS tunnels, provided there are means to deliver BGP multiprotocol NLRIs between them - routers figure out remote endpoints of tunnels from received BGP Updates;
• signaling: labels used for VPLS tunnels by remote endpoints are distributed in the same BGP Updates, this means there is no need for targeted LDP sessions between tunnel endpoints as in case of LDP signaled VPLS.

For example, if LDP signaled VPLS is used, adding new site to existing VPLS would mean configuring router that connects new site to establish tunnels with the rest of sites and also configure all other routers to establish tunnels with router connecting this new site. BGP based VPLS, if configured properly eliminates need to adjust configuration on all routers forming VPLS.

The requirement to exchange BGP NLRIs between VPLS routers means that either full mesh of BGP sessions need to be established among routers forming VPLS or route reflector must be used. In case full mesh of BGP sessions are established between VPLS routers, the benefits of BGP based VPLS over LDP signaled VPLS are questionable - when new site is added to VPLS, BGP peer configuration still needs to be entered on every router forming given VPLS. When BGP route reflector is used, adding new site to VPLS becomes more simple - router connecting new site must only peer with route reflector and no additional configuration is required on other routers. Taking into account that route reflector can also be one of routers forming VPLS, there is no need for additional separate equipment. Of course, scalability and availability concerns still must be taken into account - multiple route reflectors can be used for backup purposes as well as for distributing information load.

The drawback of running BGP based VPLS is requirement to configure BGP which requires that network administrator has at least basic understanding of BGP, its multiprotocol capabilities and route reflectors. Therefore it is advised to implement LDP signaled VPLS if amount of sites and VPLS networks is small, topology is more static - that is, benefits of using BGP are not obvious.

Note that BGP based VPLS is a method only for VPLS tunnel label exchange, it does not deal with delivery of traffic between VPLS tunnel endpoints, so general MPLS frame delivery between tunnel endpoints must be ensured as discussed in MPLSVPLS.

Suggested reading material:

• RFC 4761, Virtual Private LAN Service (VPLS) Using BGP for Auto-Discovery and Signaling
• RFC 4456, BGP Route Reflection: An Alternative to Full Mesh Internal BGP (IBGP)

Example network

Consider the same network as used for LDP signaled VPLS example in MPLSVPLS:

The requirements of customers A and B are the same - ethernet segments must be transparently connected. Taking into account simplicity of given network topology Service Provider has decided to use R5 as route reflector and to have no backup route reflector. Consider that MPLS switching is configured and running, as discussed in MPLSVPLS, but no any VPLS configuration has been applied yet. the rest of this document deals with specifics that are introduced by use of BGP for VPLS signaling.

Configuring IBGP session for VPLS signaling

At first, BGP instance must be configured, default instance can also be used:

[admin@R1] /routing bgp instance> print
Flags: X - disabled
 0   name="default" as=65530 router-id=0.0.0.0 redistribute-connected=no redistribute-static=no
     redistribute-rip=no redistribute-ospf=no redistribute-other-bgp=no out-filter=""
     client-to-client-reflection=yes ignore-as-path-len=no

To enable VPLS NLRI delivery across BGP, BGP multiprotocol capability must be used. This is enabled by specifying l2vpn in BGP peer's address-families setting.

For example, to configure BGP connection between R1 and R5, the following commands should get issued.

On R1:

[admin@R1] /routing bgp peer> add remote-address=9.9.9.5 remote-as=65530 address-families=l2vpn \
  update-source=lobridge

and on R5:

[admin@R5] /routing bgp peer> add remote-address=9.9.9.1 remote-as=65530 address-families=l2vpn \
  update-source=lobridge

BGP connection should get established between R1 and R5. This can be confirmed by:

[admin@R1] /routing bgp peer> print status
Flags: X - disabled
 0   name="peer1" instance=default remote-address=9.9.9.5 remote-as=65530 tcp-md5-key=""
     nexthop-choice=default multihop=no route-reflect=no hold-time=3m ttl=255 in-filter=""
     out-filter="" address-families=l2vpn update-source=lobridge remote-id=4.4.4.5
     local-address=9.9.9.1 uptime=3s prefix-count=0 updates-sent=0 updates-received=0
     withdrawn-sent=0 withdrawn-received=0 remote-hold-time=3m used-hold-time=3m
     used-keepalive-time=1m refresh-capability=yes state=established

There are several things to note about BGP peer configuration:

• there is no need to distribute any IP or IPv6 routes and even no need have IP or IP6 support over BGP connection at all to be able to exchange VPLS NLRIs, it is sufficient to specify address-families=l2vpn
• "loopback" addresses of routers are used as BGP peer addresses (local address is configured by means of update-source setting). BGP peer, when originating VPLS NLRI, specifies its local address as BGP NextHop (for example, in given setup R1 originating BGP NLRIs will use address 9.9.9.1 as BGP NextHop address), receiving VPLS router uses received BGP NextHop address as tunnel endpoint address and therefore uses transport label that ensures delivery to BGP NextHop. In order for penultimate hop popping to work properly, it is advised to use loopback IP address for this. See penultimate hop popping related discussion in MPLSVPLS.

Configuring Route Reflector

In its simplest sense BGP Route Reflector re-advertises received IBGP routes without changing BGP NextHop for route. This feature can be used to avoid setting up full mesh of BGP connections. Note that for router be able to operate as route reflector for VPLS NLRIs, it is not necessary for it to participate in any VPLS, it is even not necessary for it to have MPLS support. Still it is mandatory for VPLS routers to be able to establish BGP sessions with route reflector, therefore IP connectivity is a must.

Route reflector's BGP instance must be configured with client-to-client-reflection=yes setting:

[admin@R5] /routing bgp instance> print
Flags: X - disabled
 0   name="default" as=65530 router-id=0.0.0.0 redistribute-connected=no redistribute-static=no
     redistribute-rip=no redistribute-ospf=no redistribute-other-bgp=no out-filter=""
     client-to-client-reflection=yes ignore-as-path-len=no

Additionaly, peers on route reflector must be configured with route-reflect=yes setting:

[admin@R5] /routing bgp peer> print
Flags: X - disabled
 0   name="peer1" instance=default remote-address=9.9.9.1 remote-as=65530 tcp-md5-key=""
     nexthop-choice=default multihop=no route-reflect=no hold-time=3m ttl=255 in-filter=""
     out-filter="" address-families=l2vpn update-source=lobridge
[admin@R5] /routing bgp peer> set 0 route-reflect=yes
[admin@R5] /routing bgp peer> print
Flags: X - disabled
 0   name="peer1" instance=default remote-address=9.9.9.1 remote-as=65530 tcp-md5-key=""
     nexthop-choice=default multihop=no route-reflect=yes hold-time=3m ttl=255 in-filter=""
     out-filter="" address-families=l2vpn update-source=lobridge

To enable R5 to operate as route reflector, all its peers should get added with route-reflect=yes setting. So to enable proper VPLS NLRI distribution, R5 must be configured with 2 BGP peers - R1 and R4:

[admin@R5] /routing bgp peer> print status
Flags: X - disabled
 0   name="peer1" instance=default remote-address=9.9.9.1 remote-as=65530 tcp-md5-key=""
     nexthop-choice=default multihop=no route-reflect=yes hold-time=3m ttl=255 in-filter=""
     out-filter="" address-families=l2vpn update-source=lobridge remote-id=1.1.1.1
     local-address=9.9.9.5 uptime=5m55s prefix-count=0 updates-sent=0 updates-received=0
     withdrawn-sent=0 withdrawn-received=0 remote-hold-time=3m used-hold-time=3m
     used-keepalive-time=1m refresh-capability=yes state=established

 1   name="peer2" instance=default remote-address=9.9.9.4 remote-as=65530 tcp-md5-key=""
     nexthop-choice=default multihop=no route-reflect=yes hold-time=3m ttl=255 in-filter=""
     out-filter="" address-families=l2vpn update-source=lobridge remote-id=3.3.3.4
     local-address=9.9.9.5 uptime=23s prefix-count=0 updates-sent=0 updates-received=0
     withdrawn-sent=0 withdrawn-received=0 remote-hold-time=3m used-hold-time=3m
     used-keepalive-time=1m refresh-capability=yes state=established

But R1 and R4 must only peer with R5. On R1:

[admin@R1] /routing bgp peer> print status
Flags: X - disabled
 0   name="peer1" instance=default remote-address=9.9.9.5 remote-as=65530 tcp-md5-key=""
     nexthop-choice=default multihop=no route-reflect=no hold-time=3m ttl=255 in-filter=""
     out-filter="" address-families=l2vpn update-source=lobridge remote-id=4.4.4.5
     local-address=9.9.9.1 uptime=6m33s prefix-count=0 updates-sent=0 updates-received=0
     withdrawn-sent=0 withdrawn-received=0 remote-hold-time=3m used-hold-time=3m
     used-keepalive-time=1m refresh-capability=yes state=established

and on R4:

[admin@R4] /routing bgp peer> print status
Flags: X - disabled
 0   name="peer1" instance=default remote-address=9.9.9.5 remote-as=65530 tcp-md5-key=""
     nexthop-choice=default multihop=no route-reflect=no hold-time=3m ttl=255 in-filter=""
     out-filter="" address-families=l2vpn update-source=lobridge remote-id=4.4.4.5
     local-address=9.9.9.4 uptime=3s prefix-count=0 updates-sent=0 updates-received=0
     withdrawn-sent=0 withdrawn-received=0 remote-hold-time=3m used-hold-time=3m
     used-keepalive-time=1m refresh-capability=yes state=established

Using route reflector means that in order to add new site to some VPLS, e.g. connected by router Ry, would mean adding Ry as BGP peer to R5 (with route-reflect=yes setting) and adding R5 as BGP peer to Ry.

Configuring BGP signaled VPLS

Configuring ethernet bridging

BGP signalled VPLS tunnels are created dynamically when proper BGP NLRIs are received. Therefore there is no need to configure any VPLS interfaces. Still, to transparently deliver packets from ethernet segment across VPLS bridging must be configured. For example, on R1 two bridges are created, named "A" and "B" with appropriate customer-facing ethernet interfaces added to them:

[admin@R1] /interface bridge> print
Flags: X - disabled, R - running
 0  R name="lobridge" mtu=1500 arp=enabled mac-address=00:00:00:00:00:00 protocol-mode=none
      priority=0x8000 auto-mac=yes admin-mac=00:00:00:00:00:00 max-message-age=20s
      forward-delay=15s transmit-hold-count=6 ageing-time=5m

 1  R name="A" mtu=1500 arp=enabled mac-address=00:01:50:E7:00:09 protocol-mode=none 
      auto-mac=yes admin-mac=00:00:00:00:00:00 max-message-age=20s forward-delay=15s
      priority=0x8000 transmit-hold-count=6 ageing-time=5m

 2  R name="B" mtu=1500 arp=enabled mac-address=00:01:50:E7:00:08 protocol-mode=none 
      auto-mac=yes admin-mac=00:00:00:00:00:00 max-message-age=20s forward-delay=15s
      priority=0x8000 transmit-hold-count=6 ageing-time=5m
[admin@R1] /interface bridge> port print
Flags: X - disabled, I - inactive, D - dynamic
 #    INTERFACE                             BRIDGE PRIORITY PATH-COST            HORIZON
 0    ether2                                A      0x80     10                   none
 1    ether1                                B      0x80     10                   none

Configuring BGP signaled VPLS instances

Configuring BGP signaled VPLS instance makes router advertise VPLS BGP NLRI that advertises that particular router belongs to some VPLS. Upon receiving such advertisement, other members of same VPLS know to establish VPLS tunnel with this router.

To configure VPLS for customers A and B, on R1 the following commands should be issued:

[admin@R1] /interface vpls bgp-vpls> add bridge=A bridge-horizon=1 route-distinguisher=1:1 \
   site-id=1 import-route-targets=1:1 export-route-targets=1:1
[admin@R1] /interface vpls bgp-vpls> add bridge=B bridge-horizon=1 route-distinguisher=2:2 \
   site-id=1 import-route-targets=2:2 export-route-targets=2:2

Note: Since v3.20 vpls-id was replaced with separate import/export-route-targets to provide more flexibility.

route-distinguisher setting specifies value that gets attached to VPLS NLRI so that receiving routers can distinguish advertisements that may otherwise look the same. This implies that unique route-distinguisher for every VPLS must be used. It is not necessary to use the same route distinguisher for some VPLS on all routers forming that VPLS as distinguisher is not used for determining if some BGP NLRI is related to particular VPLS (Route Target attribute is used for this), but it is mandatory to have different distinguishers for different VPLSes.

export-route-targets setting is used for tagging BGP NLRI

import-route-targets setting is used to determine if BGP NLRI is related to particular VPLS

site-id setting must be unique among members of particular VPLS. It is advisable although not mandatory to allocate site-id values in as narrow range as possible as that increases efficency of BGP (for details see RFC 4761).

bridge setting specifies bridge to which dynamically created VPLS tunnels should get added.

bridge-horizon specifies horizon value to be used for ports added to bridge (see Split horizon bridging discussion in MPLSVPLS).

After configuring R4 as member of VPLS 1:1 (used for customer A) with command:

[admin@R4] /interface vpls bgp-vpls> add bridge=A bridge-horizon=1 route-distinguisher=1:1 \
   site-id=4 import-route-targets=1:1 export-route-targets=1:1

Dynamic VPLS tunnel gets created on both R1 and R4. On R1 this can be confirmed:

[admin@R1] > /interface vpls print
Flags: X - disabled, D - dynamic, R - running, B - bgp-signaled
 0 RDB name="vpls1" mtu=1500 mac-address=02:FA:33:C4:7A:A9 arp=enabled 
       disable-running-check=no remote-peer=9.9.9.4 cisco-style=no 
       cisco-style-id=0 vpls=bgp-vpls1 
[admin@R1] > /interface bridge port print
Flags: X - disabled, I - inactive, D - dynamic
 #    INTERFACE                             BRIDGE PRIORITY PATH-COST            HORIZON
 0    ether2                                A      0x80     10                   none
 1    ether1                                B      0x80     10                   none
 2  D vpls1                                 A      0x80     50                   1

Here we have confirmed also that route reflection as configured on R5 works as expected as there is no BGP peer relationship between R1 and R4.

Additionally we must configure R5 to participate in VPLS for customer A:

[admin@R5] /interface vpls bgp-vpls> add bridge=A bridge-horizon=1 route-distinguisher=1:1 \
   site-id=5 import-route-targets=1:1 export-route-targets=1:1

This causes R1 and R4 to establish additional VPLS tunnel with R5. For example on R1:

[admin@R1] > /interface vpls print
Flags: X - disabled, D - dynamic, R - running, B - bgp-signaled
 0 RDB name="vpls1" mtu=1500 mac-address=02:FA:33:C4:7A:A9 arp=enabled 
       disable-running-check=no remote-peer=9.9.9.4 cisco-style=no 
       cisco-style-id=0 vpls=bgp-vpls1 
 1 RDB name="vpls2" mtu=1500 mac-address=02:FF:B7:0E:4B:97 arp=enabled 
       disable-running-check=no remote-peer=9.9.9.5 cisco-style=no 
       cisco-style-id=0 vpls=bgp-vpls1 

And bridge port to get added with proper horizon value:

[admin@R1] > /interface bridge port print
Flags: X - disabled, I - inactive, D - dynamic
 #    INTERFACE                             BRIDGE PRIORITY PATH-COST            HORIZON
 0    ether2                                A      0x80     10                   none
 1    ether1                                B      0x80     10                   none
 2  D vpls1                                 A      0x80     50                   1
 3  D vpls2                                 A      0x80     50                   1

To complete the setup, necessary configuration for customer B VPLS should be applied to R5:

[admin@R5] /interface vpls bgp-vpls> add site-id=5 route-distinguisher=2:2 bridge=B \
    bridge-horizon=1 import-route-targets=2:2 export-route-targets=2:2

As the result we get full mesh of VPLS tunnels established, for example on R5:

[admin@R5] /interface vpls> print
Flags: X - disabled, D - dynamic, R - running, B - bgp-signaled

 0 RDB name="vpls1" mtu=1500 mac-address=02:FA:5C:28:29:D3 arp=enabled 
       disable-running-check=no remote-peer=9.9.9.1 cisco-style=no 
       cisco-style-id=0 vpls=bgp-vpls1 
 1 RDB name="vpls2" mtu=1500 mac-address=02:EA:51:31:3E:2B arp=enabled 
       disable-running-check=no remote-peer=9.9.9.4 cisco-style=no 
       cisco-style-id=0 vpls=bgp-vpls1 
 2 RDB name="vpls3" mtu=1500 mac-address=02:F6:CF:06:1E:CB arp=enabled 
       disable-running-check=no remote-peer=9.9.9.1 cisco-style=no 
       cisco-style-id=0 vpls=bgp-vpls2 

Note that remote-peer for VPLS tunnels is BGP NextHop address as received in BGP Update. For example BGP logs on R5 when receiving Update for VPLS 2:2 (customer B), say:

11:24:06 route,bgp,debug,packet UPDATE Message
11:24:06 route,bgp,debug,packet     RemoteAddress=9.9.9.1
11:24:06 route,bgp,debug,packet     MessageLength=79
11:24:06 route,bgp,debug,packet
11:24:06 route,bgp,debug,packet     PathAttributes
11:24:06 route,bgp,debug,packet         bgp-origin=INCOMPLETE
11:24:06 route,bgp,debug,packet         bgp-nexthop=9.9.9.1
11:24:06 route,bgp,debug,packet         bgp-localpref=100
11:24:06 route,bgp,debug,packet         bgp-extended-communities=RT:2:2
11:24:06 route,bgp,debug,packet
11:24:06 route,bgp,debug,packet     NLRI= rd
11:24:06 route,bgp,debug,packet         type=0
11:24:06 route,bgp,debug,packet         administrator=2
11:24:06 route,bgp,debug,packet         assigned-number=2 veId=1 veBlockOffset=0 veBlockSize=16 
                             labelBase=40

This is reflected for dynamic VPLS tunnel, where remote-peer for tunnel with export-route-targer 2:2 is 9.9.9.1. This implies that R5 uses IGP route that leads to 9.9.9.1 to decide what transport label to use. In given case there are /32 IGP routes distributed in the network by means of OSPF, therefore:

[admin@R5] /interface vpls> monitor 2 once
    remote-label: 45
     local-label: 40
   remote-status:
      igp-prefix: 9.9.9.1/32
     igp-nexthop: 4.4.4.3
  imposed-labels: 17,45

Shows that 9.9.9.1/32 route is used and immediate nexthop is 4.4.4.3. Labels attached to VPLS packets are 17 and 45 where 45 is label mapping received with BGP Update, and 17 is label assigned by R3 for prefix 9.9.9.1/32:

[admin@R5] > /mpls remote-bindings print
Flags: X - disabled, A - active, D - dynamic
 #    DST-ADDRESS        NEXTHOP         LABEL      PEER
 ...
14 AD 9.9.9.1/32         4.4.4.3         17         9.9.9.3:0
 ...

See also

MPLSVPLS