In this post we will set up our Remote Tunnel Endpoints (RTEPs) to allow us to tunnel our overlay traffic across sites.
This post is part 5 of a multipart series. Find the other parts here:
- Part 1: Lab Setup and Overview
- Part 2: Site A Build
- Part 3: Automated Site B Build
- Part 4: NSX-T Site Federation
- Part 5: This Part: Remote Tunnel Endpoints
- Part 6: Federated Tier-0 Gateway
- Part 7: Federated Tier-1 Gateways
- Part 8: Egress Traffic and MEDdling with BGP
As a reminder, in this series we will be building the following lab:
(Click image to zoom in)
Overview
What is a Remote Tunnel End Point or RTEP?
Just like the host and edge TEPs, NSX-T Geneve traffic needs to be encapsulated and de-encapsulated by a Tunnel End Point (TEP). RTEPs are used for cross site traffic from Edge node to Edge node. If we want to pass encapsulated overlay traffic from one site to another site, we are going to need some RTEPs.
OK, so let’s get some RTEPs configured!
Check MTU
As discussed in the VMware article NSX-T Network Requirements and Sizing for NSX-T Workload Domains:
Yep, we need to have a cross site maximum transmission unit (MTU) of at least 1600. So to save heart ache further down the road, let’s double check our site MTU settings at each site. Let’s also test our cross site MTU.
Open NSX-T Global Manager, select a site and navigate to System > Fabric > Settings. Confirm that both Tunnel End Point (TEP) and Remote Tunnel End Point (RTEP) settings are set to 1700 and click Check Now:
If all OK, the overall MTU status should return “Consistent”. If not, make necessary adjustments and check again. Repeat for all other sites.
Next let’s check the cross site network MTU. Obviously the amount and complexity of testing will very much depend on the complexity of network between your NSX-T between sites. Luckily for us, as this is a lab and as you can see from the diagram above we have just the one device between our NSX-T sites; LABROUTER.
As we are using an OPNsense for our LABROUTER, confirmation of the RTEP VLAN MTU configuration is easy. Log in to OPNsense, select Interfaces > Diagnostics > Netstat and lets look at our Site A RTEP VLAN interface, SITE_A_RMOTE_TRANSPORT_VL13 As defined in Part 1:
Looks good. Repeat for Site B RTEP VLAN interface SITE_B_RMOTE_TRANSPORT_VL23.
RTEP IP Pools
Next let’s create some Remote Tunnel End Point IP pools.
Open NSX-T Global Manager, select a site, then select System > Networking > IP Address Pools > Add IP Address Pool.
Name the Pool Site-A-RTEP-Pool, click Set > Add Subnet > IP Ranges.
As per Site A IP Allocation, set the IP range to 192.168.13.2-192.168.13.254, the CIDR to 192.168.13.0/24, the Gateway IP to 192.168.13.1 and click Add:
Click Apply and Save. When complete you should have the following:
Again, repeat for Site B.
Configure Inter-Location Communication
With that all done, lets enable come inter-site comms.
Open NSX-T Global Manager, select the Global Manager, then select Location Manager. From there select a location and click Networking.
Confirm that the correct edge cluster is selected and click Configure:
Select the edge cluster again and complete the RTEP Configuration:
Click Save to complete. Repeat for Site B.
Configuration Confirmation
We will build our Tier 0 and Tier 1 stretched gateways in part 6. Until then, let’s confirm that we are ready for them.
Open NSX-T Global Manager and select System > System Overview. Scroll down to the locations:
Lets look closer at our RTEP “unknown” status:
So our RTEP status looks good. Furthermore, clicking the i tells us that we are indeed ready:
Conclusion and Wrap Up
So there we have it. Our federated NSX-T sites now have their remote tunnel endpoints. We still have to create our Global Tier 0 and Tier 1 Logical routers before we can hook any VMs into our NSX-T build. We will look at that in a later part of this series.
This was part 5 of a multipart series. Find the other parts here:
- Part 1: Lab Setup and Overview
- Part 2: Site A Build
- Part 3: Automated Site B Build
- Part 4: NSX-T Site Federation
- Part 5: This Part: Remote Tunnel Endpoints
- Part 6: Federated Tier-0 Gateway
- Part 7: Federated Tier-1 Gateways
- Part 8: Egress Traffic and MEDdling with BGP
Look out for future parts coming soon!
-Chris