VXLAN EVPN Multisite Setup – Part 4

Quick Recap
Part 1 – Setup simple L2 VXLAN within a single site
Part 2 – Setup L3 VNI within a single site
Part 3 – Setup multisite between 2 sites using a route server
Part 4, we will be configuring external connectivity via the shared border which is on 9K-10 below


9K-10 already has a BGP peering with the ISP router which is advertising a default route to 9K-10. First we need to setup VNI 1000 and the Tenant1 routing table so we can import/export routes between the VRF and the global table. The sites will be configured to advertise an aggregate of 192.168.0.0/16 + the host routes to this shared border. The shared border will then export that aggregate to the ISP, while also advertising a default route to the sites.

We’ll start with enabling the features required

9K-10(config)# feature vn-segment-vlan-based
9K-10(config)# feature nv overlay
9K-10(config)# nv overlay evpn
9K-10(config)# feature interface-vlan

Next, create the L3 VNI and VRF

9K-10(config)# vlan 1000
9K-10(config-vlan)# vn-segment 1000
!
9K-10(config-vlan)# vrf context TENANT1
9K-10(config-vrf)# vni 1000
9K-10(config-vrf)# rd auto
9K-10(config-vrf)# add ipv4 un
9K-10(config-vrf-af-ipv4)# route-target both auto ev
!
9K-10(config-vrf-af-ipv4)# int vlan 1000
9K-10(config-if)# vrf mem TENANT1
Warning: Deleted all L3 config on interface Vlan1000
9K-10(config-if)# ip for
9K-10(config-if)# ip forward
9K-10(config-if)# no shut

Configure the NVE interface

9K-10(config-if)# int nve1
9K-10(config-if-nve)# source-interface lo1
9K-10(config-if-nve)# host-reachability protocol bgp
9K-10(config-if-nve)# mem vni 1000 associate-vrf
9K-10(config-if-nve-vni)# no shut

Now, add the loopbacks for advertisement to the sites

9K-10(config-if-nve-vni)# router bgp 1000
9K-10(config-router)# add ipv4 un
9K-10(config-router-af)# net 10.10.255.1/32
9K-10(config-router-af)# net 10.10.254.1/32

Add R2 as a BGP neighbor to learn the loopbacks from the sites

9K-10(config-router-af)# nei 10.0.104.1
9K-10(config-router-neighbor)# remote-as 65000
9K-10(config-router-neighbor)# add ipv4 un
! Verify
9K-10(config-router-neighbor-af)# sh bgp ip un sum
......
Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.104.1      4 65000       9       4       17    0    0 00:00:01 10
10.0.105.1      4 65000      11      10       17    0    0 00:06:15 1

Now create a prefix-list and route-map to match on the 192.168.0.0/16 aggregate that the sites will soon advertise. We’ll then attach this to the vrf as an export map

9K-10(config-router-neighbor-af)# ip prefix-list TENANT1-AGGREGATE permit 192.168.0.0/16
9K-10(config)# route-map TENANT1-AGGREGATE permit 10
9K-10(config-route-map)# match ip add pre TENANT1-AGGREGATE
!
9K-10(config-route-map)# vrf context TENANT1
9K-10(config-vrf)# add ipv4 un
9K-10(config-vrf-af-ipv4)# export vrf default map TENANT1-AGGREGATE allow-vpn

While we are in there, let’s also configure the static default route pointing to the ISP. We can then advertise that to the sites

9K-10(config-vrf-af-ipv4)# ip route 0.0.0.0/0 10.0.105.1 vrf default

Add the route server peering + jump to the route server and add the shared border peering as well

9K-10(config-if-nve)# router bgp 1000
9K-10(config-router)# nei 10.0.106.2
9K-10(config-router-neighbor)# remote-as 1001
9K-10(config-router-neighbor)# ebgp-multihop 5
9K-10(config-router-neighbor)# update-source lo0
9K-10(config-router-neighbor)# add l2 ev
9K-10(config-router-neighbor-af)# send-community
both       extended   standard
9K-10(config-router-neighbor-af)# send-community both
9K-10(config-router-neighbor-af)# rewrite-evpn-rt-asn
!
! On route server
9K-9(config)# router bgp 1001
9K-9(config-router)# nei 10.10.255.1
9K-9(config-router-neighbor)# remote-as 1000
9K-9(config-router-neighbor)# ebgp-multihop 5
9K-9(config-router-neighbor)# add l2 evpn
9K-9(config-router-neighbor-af)# send-community extended
9K-9(config-router-neighbor-af)# route-map NEXT-HOP-UNCHANGED out
9K-9(config-router-neighbor-af)# rewrite-evpn-rt-asn

Verify the peering on shared border

9K-10(config-router-neighbor-af)# do sh bgp l2 ev sum
.......
Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.106.2      4  1001      27      12       12    0    0 00:01:15 4

Let’s take a quick jump to site 1 to configure the aggregate

9K-1(config)# router bgp 100
9K-1(config-router)# vrf TENANT1
9K-1(config-router-vrf)# add ipv4 un
9K-1(config-router-vrf-af)# aggregate-address 192.168.0.0/16

Now let’s check out the routing table for Tenant1 on the shared border

K-10(config-router-neighbor-af)# end
9K-10# sh ip route vrf TENANT1
........
0.0.0.0/0, ubest/mbest: 1/0
    *via 10.0.105.1%default, [1/0], 00:05:48, static
192.168.0.0/16, ubest/mbest: 1/0
    *via 10.2.254.1%default, [20/0], 00:01:46, bgp-1000, external, tag 1001 (evp
n) segid: 1000 tunnelid: 0xa02fe01 encap: VXLAN

192.168.100.12/32, ubest/mbest: 1/0
    *via 10.1.253.1%default, [20/0], 00:01:46, bgp-1000, external, tag 1001 (evp
n) segid: 1000 tunnelid: 0xa01fd01 encap: VXLAN

192.168.100.15/32, ubest/mbest: 1/0
    *via 10.2.253.1%default, [20/0], 00:01:46, bgp-1000, external, tag 1001 (evp
n) segid: 1000 tunnelid: 0xa02fd01 encap: VXLAN

192.168.200.13/32, ubest/mbest: 1/0
    *via 10.1.253.1%default, [20/0], 00:01:46, bgp-1000, external, tag 1001 (evp
n) segid: 1000 tunnelid: 0xa01fd01 encap: VXLAN

Now we can see that the aggregate is working and we should be receiving them both from site 1 and site 2

9K-10# sh bgp l2 ev
......
   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 10.1.255.1:3
*>e[5]:[0]:[0]:[16]:[192.168.0.0]/224
                      10.1.254.1                                     0 1001 100
i
.....
Route Distinguisher: 10.2.255.1:3
*>e[5]:[0]:[0]:[16]:[192.168.0.0]/224
                      10.2.254.1                                     0 1001 200
i

You can see from looking at the routing table that it’s only using the aggregate from site 2. Let’s configure multipath, but also relax the best path algorithm to allow multipath from different AS numbers

9K-10(config)# router bgp 1000
9K-10(config-router)# vrf TENANT1
9K-10(config-router-vrf)# add ipv4 un
9K-10(config-router-vrf-af)# maximum-paths 2
9K-10(config-router-vrf-af)# bestpath as-path multipath-relax
!
9K-10(config-router-vrf-af)# sh ip route 192.168.0.0/16 vrf TENANT1
.....
192.168.0.0/16, ubest/mbest: 2/0
    *via 10.1.254.1%default, [20/0], 00:00:07, bgp-1000, external, tag 1001 (evp
n) segid: 1000 tunnelid: 0xa01fe01 encap: VXLAN
    *via 10.2.254.1%default, [20/0], 00:00:19, bgp-1000, external, tag 1001 (evp
n) segid: 1000 tunnelid: 0xa02fe01 encap: VXLAN

Now let’s make sure the ISP has received the aggregate

R10#sh ip route bgp
.......
      10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks
B        10.10.254.1/32 [20/0] via 10.0.105.2, 00:17:01
B        10.10.255.1/32 [20/0] via 10.0.105.2, 00:17:04
B     192.168.0.0/16 [20/0] via 10.0.105.2, 00:07:51

Let’s also tell BGP to advertise the default to the sites, and then confirm at the sites

9K-10(config)# router bgp 1000
9K-10(config-router)# vrf TENANT1
9K-10(config-router-vrf)# add ipv4 un
9K-10(config-router-vrf-af)# net 0.0.0.0/0
!
9K-1# sh ip route vrf TENANT1
......
0.0.0.0/0, ubest/mbest: 1/0
    *via 10.10.254.1%default, [20/0], 00:00:59, bgp-100, external, tag 1001 (evp
n) segid: 1000 tunnelid: 0xa0afe01 encap: VXLAN
!
9K-4# sh ip route vrf TENANT1
.......
0.0.0.0/0, ubest/mbest: 1/0
    *via 10.10.254.1%default, [20/0], 00:01:21, bgp-200, external, tag 1001 (evp
n) segid: 1000 tunnelid: 0xa0afe01 encap: VXLAN
!
! Let's also verify the leafs are getting it
9K-2# sh ip route vrf TENANT1
.......
0.0.0.0/0, ubest/mbest: 1/0
    *via 10.1.253.1%default, [200/0], 00:01:40, bgp-100, internal, tag 1001 (evp
n) segid: 1000 tunnelid: 0xa01fd01 encap: VXLAN

Finally, let’s see if the PCs at each site can ping the 4.2.2.2 loopback on the ISP router

PC-2> ping 4.2.2.2
84 bytes from 4.2.2.2 icmp_seq=1 ttl=251 time=758.689 ms
84 bytes from 4.2.2.2 icmp_seq=2 ttl=251 time=220.428 ms
!
PC-3> ping 4.2.2.2
84 bytes from 4.2.2.2 icmp_seq=1 ttl=251 time=81.813 ms
84 bytes from 4.2.2.2 icmp_seq=2 ttl=251 time=164.076 ms
!
PC-5> ping 4.2.2.2
84 bytes from 4.2.2.2 icmp_seq=1 ttl=252 time=76.222 ms
84 bytes from 4.2.2.2 icmp_seq=2 ttl=252 time=146.690 ms

All done.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.