Configure VXLAN with Ansible Part 3
This is the third and final part of the VXLAN Ansible configuration series. If you have followed the previous sections, you should already have the base, and underlay configuration applied and are now ready to configure VXLAN with BGP EVPN. As with the previous configuration sections, I will be creating another role that will do the following: Finish off the PIM RP configuration, configure the NVE interfaces, the VLAN's and VLAN to VNI mappings and configure BGP, VRF's and the Anycast gateways for all Spine, Leaf and Border Gateway switches.
There is quite a lot of configuration to be applied in this last stage which is why I've split these into different roles and tasks because I find it easier to manage. As with the previous parts to this guide, I have created a new role for this play called dcb_vxlan. Within the dcb_vxlan role folder, I then created the following folder structure and nine YML files.
(ansible root)
roles
- dcb_vxlan
- tasks
- main.yml
- dcb_sw_leaf.yml
- dcb_sw_bgw.yml
- dcb_vxlan.yml
- dcb_sw_config_vrf.yml
- dcb_bgw_config_vrf.yml
- dcb_leaf_evpn.yml
- dcb_leaf_anycast_svi_config.yml
You don't need to copy this structure, you could simply use the main.yml file and place all of your tasks in there or do whatever you like really, I just find this easier to follow. Once you have your file/folder structure all setup the way you want it, start by configuring the main.yml file. I've used this file to call the other task files in order and to specify which hosts to run the tasks on and when. Again, you can do this however you want, I just find it easier to manage and run tasks this way.
main.yml
- name: Configure leaf switch ready for VXLAN
include_role:
name: dcb_vxlan
tasks_from: dcb_sw_leaf.yml
when: inventory_hostname in groups['dcb_leafs']
- name: Configure BGW switch ready for VXLAN
include_role:
name: dcb_vxlan
tasks_from: dcb_sw_bgw.yml
when: inventory_hostname in groups['dcb_bgw']
- name: Configure basic BGP settings on all switches
include_role:
name: dcb_vxlan
tasks_from: dcb_base_bgp.yml
- name: Confgure basic VRFs and BGP for all leaf/BGW switches
include_role:
name: dcb_vxlan
tasks_from: dcb_sw_config_vrf.yml
when: (inventory_hostname in groups['dcb_leafs']) or (inventory_hostname in groups['dcb_bgw'])
- name: Configure VRFs and route-maps on BGW switch
include_role:
name: dcb_vxlan
tasks_from: dcb_bgw_config_vrf.yml
when: inventory_hostname in groups['dcb_bgw']
- name: Configure EVPN to VNI mapping on Leaf switches
include_role:
name: dcb_vxlan
tasks_from: dcb_leaf_evpn.yml
when: inventory_hostname in groups['dcb_leafs']
- name: Configure Leaf switch anycast gateway and SVIs
include_role:
name: dcb_vxlan
tasks_from: dcb_leaf_anycast_svi_config.yml
when: inventory_hostname in groups['dcb_leafs']
- name: SAVE RUN CONFIG TO STARTUP CONFIG
cisco.nxos.nxos_config:
save_when: alwaysWith the main.yml file created, next create each of the task files in the order they will be run, so start with the dcb_sw_leaf.yml file. This play configures the first parts of the VXLAN settings such as the VLANs, VNI, VLAN to VNI mapping, NVE interface and the required VRF settings ready for BGP EVPN
dcb_sw_leaf.yml
- name: Configure VLAN to VNI mapping
cisco.nxos.nxos_vlans:
config:
- vlan_id: "{{ item.vlan_id }}"
name: "{{ item.name }}"
mapped_vni: "{{ item.vni_id }}"
with_items:
- "{{ vlan_l2vni }}"
- "{{ vlan_l3vni }}"
- name: Create the NVE Interface
cisco.nxos.nxos_interfaces:
config:
- name: "{{ nve_int }}"
enabled: true
- name: Configure the NVE Interface
cisco.nxos.nxos_vxlan_vtep:
interface: "{{ nve_int }}"
description: VXLAN VTEP Interface
source_interface: "{{ nve_src_int }}"
host_reachability: true
state: present
- name: Configure NVE L2 VNI mapping
cisco.nxos.nxos_vxlan_vtep_vni:
interface: "{{ nve_int }}"
vni: "{{ item.vni_id }}"
multicast_group: "{{ item.mcast_grp }}"
loop: "{{ vlan_l2vni }}"
- name: Configure SVI's for VNIs
cisco.nxos.nxos_interfaces:
config:
- name: "Vlan{{ item.vlan_id }}"
enabled: true
with_items:
- "{{ vlan_l2vni }}"
- "{{ vlan_l3vni }}"
- name: Configure SVI IP Address
cisco.nxos.nxos_l3_interfaces:
config:
- name: "Vlan{{ item.vlan_id }}"
ipv4:
- address: "{{ item.addr }}/{{ item.mask }}"
loop: "{{ vlan_l2vni }}"
- name: Enable IGMP snooping for VXLAN
cisco.nxos.nxos_command:
commands:
- conf t
- ip igmp snooping vxlan
- name: Configure VRFs
cisco.nxos.nxos_vrf:
vrf: "{{ item.vrf }}"
vni: "{{ item.vni_id }}"
rd: auto
state: present
loop: "{{ vrfs }}"
- name: Configure VRF address-family
cisco.nxos.nxos_vrf_af:
vrf: "{{ item.vrf }}"
afi: "{{ item.afi }}"
route_target_both_auto_evpn: true
state: present
loop: "{{ vrfs }}"
- name: Configure NVE Interface L3VNI mapping
cisco.nxos.nxos_vxlan_vtep_vni:
interface: "{{ nve_int }}"
vni: "{{ item.vni_id }}"
assoc_vrf: true
loop: "{{ vlan_l3vni }}"The border gateway configuration is much the same as a leaf switch as it serves a similar purpose. In fact the only difference is that in this lab, the border gateway switch doesn't have any SVI's created on it. If you need to add any SVI's to your border gateway, you can copy the tasks Configure SVI's for VNIs and Configure SVI IP Address from the leaf switch play above.
dcb_sw_bgw.yml
- name: Configure VLAN to VNI mapping
cisco.nxos.nxos_vlans:
config:
- vlan_id: "{{ item.vlan_id }}"
name: "{{ item.name }}"
mapped_vni: "{{ item.vni_id }}"
with_items:
- "{{ vlan_l2vni }}"
- "{{ vlan_l3vni }}"
- name: Create the NVE Interface
cisco.nxos.nxos_interfaces:
config:
- name: "{{ nve_int }}"
enabled: true
- name: Configure the NVE Interface
cisco.nxos.nxos_vxlan_vtep:
interface: "{{ nve_int }}"
description: VXLAN VTEP Interface
source_interface: "{{ nve_src_int }}"
host_reachability: true
state: present
- name: Configure NVE L2 VNI mapping
cisco.nxos.nxos_vxlan_vtep_vni:
interface: "{{ nve_int }}"
vni: "{{ item.vni_id }}"
multicast_group: "{{ item.mcast_grp }}"
loop: "{{ vlan_l2vni }}"
- name: Enable IGMP snooping for VXLAN
cisco.nxos.nxos_command:
commands:
- conf t
- ip igmp snooping vxlan
- name: Configure VRFs
cisco.nxos.nxos_vrf:
vrf: "{{ item.vrf }}"
vni: "{{ item.vni_id }}"
rd: auto
state: present
loop: "{{ vrfs }}"
- name: Configure VRF address-family
cisco.nxos.nxos_vrf_af:
vrf: "{{ item.vrf }}"
afi: "{{ item.afi }}"
route_target_both_auto_evpn: true
state: present
loop: "{{ vrfs }}"
- name: Configure NVE Interface L3VNI mapping
cisco.nxos.nxos_vxlan_vtep_vni:
interface: "{{ nve_int }}"
vni: "{{ item.vni_id }}"
assoc_vrf: true
loop: "{{ vlan_l3vni }}"Once you have your leaf and border gateway switch tasks created for basic the VXLAN configuration, the next step is to start configuring BGP. BGP is used as the routing protocol for VXLAN EVPN as it has the capability of transporting the required address family NLRI information that is VXLAN, ie l2vpn evpn NLRI. This configuration is applied to all switches in your VXLAN fabric to ensure full reachability. The BGP configuration for this lab uses route-reflector to ensure a full iBGP mesh design with the leaf and border gateway switches all configured as route-reflector clients.
When configuring BGP, I prefer to use templates as it simplifies the configuration and ensures that each BGP peer has the required configuration settings.
dcb_base_bgp.yml
- name: Configure global BGP settings
cisco.nxos.nxos_bgp_global:
config:
as_number: "{{ asn }}"
router_id: "{{ rid }}"
log_neighbor_changes: true
state: merged
- name: Configure BGP Templates
cisco.nxos.nxos_command:
commands:
- conf t
- router bgp {{ asn }}
- template peer {{ item.template_name }}
- remote-as {{ item.remote_as }}
- update-source {{ item.update_source }}
- address-family {{ item.afi }} {{ item.safi }}
- soft-reconfiguration inbound
- address-family {{ item.evpn_afi }} {{ item.evpn_safi }}
- send-community both
- end
loop: "{{ bgp_templates }}"
- name: Configure basic address family and any networks to be advertised
cisco.nxos.nxos_bgp_address_family:
config:
as_number: "{{ asn }}"
address_family:
- afi: "{{ item.afi }}"
safi: "{{ item.safi }}"
networks:
- prefix: "{{ item.prefix }}/{{ item.mask}}"
loop: "{{ bgp_networks }}"
when: bgp_networks is defined
- name: Configure BGP Peers
cisco.nxos.nxos_bgp_global:
config:
as_number: "{{ asn }}"
neighbors:
- neighbor_address: "{{ item.neighbour }}"
inherit:
peer: "{{ item.template_name }}"
state: merged
loop: "{{ bgp_neighbours }}"
- name: Enable IGMP snooping for VLAN
cisco.nxos.nxos_command:
commands:
- conf t
- ip igmp snooping vxlanNow that the base BGP configuration has been applied, it's time to start creating the VRF's, NVE interface, VLAN's and VNI's that will be required as part of the VXLAN configuration. Within a VXLAN fabric, VRF's only need to be configured on the required leaf and border switches and only the VRFs that will actually be required on that leaf/bgw. Note that for layer 3 routing between VRF's, a single L3 VLAN/VNI is required. This VLAN does not require an IP address, it just needs to be configured to ensure routing between subnets within the VRF.
dcb_sw_config_vrf.yml
- name: Configure VRFs
cisco.nxos.nxos_vrf:
vrf: "{{ item.vrf }}"
vni: "{{ item.vni_id }}"
rd: auto
state: present
loop: "{{ vrfs }}"
- name: Configure VRF address-family
cisco.nxos.nxos_vrf_af:
vrf: "{{ item.vrf }}"
afi: ipv4
route_target_both_auto_evpn: true
state: present
loop: "{{ vrfs }}"
- name: Configure NVE Interface L3VNI mapping
cisco.nxos.nxos_vxlan_vtep_vni:
interface: nve1
vni: "{{ item.vni_id }}"
assoc_vrf: true
loop: "{{ vlan_l3vni }}"
- name: Configure VLAN to VNI mapping
cisco.nxos.nxos_vlans:
config:
- vlan_id: "{{ item.vlan_id }}"
name: "{{ item.name }}"
mapped_vni: "{{ item.vni_id }}"
with_items:
- "{{ vlan_l2vni }}"
- "{{ vlan_l3vni }}"
- name: Configure VRF's under BGP
cisco.nxos.nxos_bgp_address_family:
config:
as_number: "{{ asn }}"
address_family:
- afi: "{{ item.afi }}"
safi: "{{ item.safi }}"
vrf: "{{ item.vrf }}"
advertise_l2vpn_evpn: true
loop: "{{ vrfs }}" In this lab, the border gateway connects directly to an external non fabric network, this next task configures the border gateway interfaces and BGP routing to ensure that those networks are reachable and routing information is exchanged.
dcb_bgw_config_vrf.yml
- name: Associate VRFs to Ethernet Interfaces
cisco.nxos.nxos_vrf_interface:
vrf: "{{ item.vrf }}"
interface: "{{ item.int }}"
with_items:
- "{{ vrf_interfaces }}"
- name: Confgure IP Forwarding for Ethernet Interfaces
cisco.nxos.nxos_interfaces:
config:
- name: "{{ item.int }}"
ip_forward: true
loop: "{{ vrf_interfaces }}"
- name: Configure Interface IP addresses
cisco.nxos.nxos_l3_interfaces:
config:
- name: "{{ item.int }}"
ipv4:
- address: "{{ item.addr }}/{{ item.mask }}"
tag: "{{ item.tag }}"
loop: "{{ vrf_interfaces }}"
- name: Create route-map to redistribute connected
cisco.nxos.nxos_route_maps:
config:
- route_map: "{{ route_map_name }}"
entries:
- sequence: 10
action: permit
description: Redistribute tagged connected subnets
match:
tags: "{{ item.tag }}"
with_items: "{{ vrf_interfaces }}"
- name: Configure VRF's under BGP
cisco.nxos.nxos_bgp_address_family:
config:
as_number: "{{ asn }}"
address_family:
- afi: "{{ item.afi }}"
safi: "{{ item.safi }}"
vrf: "{{ item.vrf }}"
advertise_l2vpn_evpn: true
redistribute:
- protocol: direct
route_map: "{{ route_map_name }}"
loop: "{{ vrfs }}" Now that we have most of the base overlay configuration started, the next task configures the EVPN L2 VNI evpn settings on the leaf switches to ensure routing information is exchanged. The VNI is a 1 to 1 mapping of a VLAN to a VNI number and is used when advertising host information within the VXLAN fabric.
dcb_leaf_evpn.yml
- name: Leaf Switch EVPN VNI
cisco.nxos.nxos_evpn_vni:
vni: "{{ item.vni_id }}"
route_distinguisher: auto
route_target_import: auto
route_target_export: auto
loop: "{{ vlan_l2vni }}"The last play for this lab, is used to configure the anycast gateway settings on all leaf switches within the fabric. The anycast gateway mac is required on all leaf switches to ensure that no matter where an end host is physically in the network or which leaf switch it is connected to, the next hop information remains the same. This ensures network reachability anywhere within the fabric for all hosts. This play also configures the SVI's and the route-map that is used to redistribute the SVI subnets throughout the VXLAN fabric and finally enables the l2vpn address family within BGP to ensure that the correct NLRI is advertised for all L2 and L3 host information.
dcb_leaf_anycast_swi_config.yml
- name: Configure Anycast Gateway MAC
cisco.nxos.nxos_overlay_global:
anycast_gateway_mac: "2020.0000.00aa"
- name: Configure SVI's for VNIs
cisco.nxos.nxos_interfaces:
config:
- name: "Vlan{{ item.vlan_id }}"
enabled: true
with_items:
- "{{ vlan_l2vni }}"
- "{{ vlan_l3vni }}"
- name: Associate VRFs to SVI
cisco.nxos.nxos_vrf_interface:
vrf: "{{ item.vrf }}"
interface: "vlan{{ item.vlan_id }}"
with_items:
- "{{ vlan_l2vni }}"
- "{{ vlan_l3vni }}"
- name: Enable anycast gateway under L2VNI SVI
cisco.nxos.nxos_interfaces:
config:
- name: "Vlan{{ item.vlan_id }}"
fabric_forwarding_anycast_gateway: true
loop: "{{ vlan_l2vni }}"
- name: Confgure IP Forwarding under L3VNI SVI
cisco.nxos.nxos_interfaces:
config:
- name: "vlan{{ item.vlan_id }}"
ip_forward: true
loop: "{{ vlan_l3vni }}"
- name: Configure SVI IP Address
cisco.nxos.nxos_l3_interfaces:
config:
- name: "Vlan{{ item.vlan_id }}"
ipv4:
- address: "{{ item.addr }}/{{ item.mask }}"
tag: "{{ item.tag }}"
loop: "{{ vlan_l2vni }}"
- name: Create route-map to redistribute connected
cisco.nxos.nxos_route_maps:
config:
- route_map: "{{ route_map_name }}"
entries:
- sequence: 10
action: permit
description: Redistribute tagged connected subnets
match:
tags: "{{ item.tag }}"
with_items: "{{ vlan_l2vni }}"
- name: Configure VRF's under BGP
cisco.nxos.nxos_bgp_address_family:
config:
as_number: "{{ asn }}"
address_family:
- afi: "{{ item.afi }}"
safi: "{{ item.safi }}"
vrf: "{{ item.vrf }}"
advertise_l2vpn_evpn: true
redistribute:
- protocol: direct
route_map: "{{ route_map_name }}"
loop: "{{ vrfs }}" And that's it, this is all that's required to configure a basic VXLAN fabric using Ansible. It's not time to run the play. Below is a shortened output from my own lab.
TASK [dcb_vxlan : Configure VLAN to VNI mapping] **********************************************************************************************************************************************************
ok: [wrlablfswb01] => (item={'vlan_id': 3010, 'name': 'MGMT_SVRS', 'vni_id': 13010, 'addr': '10.199.10.1', 'mask': 24, 'mcast_grp': '239.0.0.10', 'vrf': 'Prod', 'tag': 54321})
ok: [wrlablfswb02] => (item={'vlan_id': 3010, 'name': 'MGMT_SVRS', 'vni_id': 13010, 'addr': '10.199.10.1', 'mask': 24, 'mcast_grp': '239.0.0.10', 'vrf': 'Prod', 'tag': 54321})
ok: [wrlablfswb03] => (item={'vlan_id': 3010, 'name': 'MGMT_SVRS', 'vni_id': 13010, 'addr': '10.199.10.1', 'mask': 24, 'mcast_grp': '239.0.0.10', 'vrf': 'Prod', 'tag': 54321})
ok: [wrlablfswb01] => (item={'vlan_id': 3014, 'name': 'COMMON_SVRS', 'vni_id': 13014, 'addr': '10.199.14.1', 'mask': 24, 'mcast_grp': '239.0.0.14', 'vrf': 'Prod', 'tag': 54321})
ok: [wrlablfswb02] => (item={'vlan_id': 3014, 'name': 'COMMON_SVRS', 'vni_id': 13014, 'addr': '10.199.14.1', 'mask': 24, 'mcast_grp': '239.0.0.14', 'vrf': 'Prod', 'tag': 54321})
ok: [wrlablfswb03] => (item={'vlan_id': 3014, 'name': 'COMMON_SVRS', 'vni_id': 13014, 'addr': '10.199.14.1', 'mask': 24, 'mcast_grp': '239.0.0.14', 'vrf': 'Prod', 'tag': 54321})
changed: [wrlablfswb01] => (item={'vlan_id': 3100, 'name': 'LAB_WORKSTATIONS', 'vni_id': 13100, 'addr': '10.199.14.1', 'mask': 24, 'mcast_grp': '239.0.0.14', 'vrf': 'Prod', 'tag': 54321})
ok: [wrlablfswb02] => (item={'vlan_id': 3100, 'name': 'LAB_WORKSTATIONS', 'vni_id': 13100, 'addr': '10.199.14.1', 'mask': 24, 'mcast_grp': '239.0.0.14', 'vrf': 'Prod', 'tag': 54321})
ok: [wrlablfswb03] => (item={'vlan_id': 3100, 'name': 'LAB_WORKSTATIONS', 'vni_id': 13100, 'addr': '10.199.14.1', 'mask': 24, 'mcast_grp': '239.0.0.14', 'vrf': 'Prod', 'tag': 54321})
ok: [wrlablfswb02] => (item={'vlan_id': 3000, 'name': 'L3_VNI', 'vni_id': 13000, 'vrf': 'Prod'})
ok: [wrlablfswb01] => (item={'vlan_id': 3000, 'name': 'L3_VNI', 'vni_id': 13000, 'vrf': 'Prod'})
ok: [wrlablfswb03] => (item={'vlan_id': 3000, 'name': 'L3_VNI', 'vni_id': 13000, 'vrf': 'Prod'})
ok: [wrlablfswb02] => (item={'vlan_id': 3012, 'name': 'WEB_SVRS', 'vni_id': 23012, 'addr': '10.199.12.1', 'mask': 24, 'mcast_grp': '239.0.0.12', 'vrf': 'Servers', 'tag': 54321})
ok: [wrlablfswb01] => (item={'vlan_id': 3012, 'name': 'WEB_SVRS', 'vni_id': 23012, 'addr': '10.199.12.1', 'mask': 24, 'mcast_grp': '239.0.0.12', 'vrf': 'Servers', 'tag': 54321})
ok: [wrlablfswb03] => (item={'vlan_id': 3012, 'name': 'WEB_SVRS', 'vni_id': 23012, 'addr': '10.199.12.1', 'mask': 24, 'mcast_grp': '239.0.0.12', 'vrf': 'Servers', 'tag': 54321})
TASK [dcb_vxlan : Create the NVE Interface] ***************************************************************************************************************************************************************
ok: [wrlablfswb02]
ok: [wrlablfswb01]
ok: [wrlablfswb03]
TASK [dcb_vxlan : Configure the NVE Interface] ************************************************************************************************************************************************************
ok: [wrlablfswb01]
ok: [wrlablfswb02]
ok: [wrlablfswb03]
------------------- OMITTED ------------------------
TASK [dcb_vxlan : Configure SVI IP Address] ***************************************************************************************************************************************************************
changed: [wrlablfswb03] => (item={'vlan_id': 3010, 'name': 'MGMT_SVRS', 'vni_id': 13010, 'addr': '10.199.10.1', 'mask': 24, 'mcast_grp': '239.0.0.10', 'vrf': 'Prod', 'tag': 54321})
changed: [wrlablfswb03] => (item={'vlan_id': 3014, 'name': 'COMMON_SVRS', 'vni_id': 13014, 'addr': '10.199.14.1', 'mask': 24, 'mcast_grp': '239.0.0.14', 'vrf': 'Prod', 'tag': 54321})
changed: [wrlablfswb03] => (item={'vlan_id': 3100, 'name': 'LAB_WORKSTATIONS', 'vni_id': 13100, 'addr': '10.199.14.1', 'mask': 24, 'mcast_grp': '239.0.0.14', 'vrf': 'Prod', 'tag': 54321})
TASK [dcb_vxlan : Create route-map to redistribute connected] *********************************************************************************************************************************************
ok: [wrlablfswb03] => (item={'vlan_id': 3010, 'name': 'MGMT_SVRS', 'vni_id': 13010, 'addr': '10.199.10.1', 'mask': 24, 'mcast_grp': '239.0.0.10', 'vrf': 'Prod', 'tag': 54321})
ok: [wrlablfswb03] => (item={'vlan_id': 3014, 'name': 'COMMON_SVRS', 'vni_id': 13014, 'addr': '10.199.14.1', 'mask': 24, 'mcast_grp': '239.0.0.14', 'vrf': 'Prod', 'tag': 54321})
ok: [wrlablfswb03] => (item={'vlan_id': 3100, 'name': 'LAB_WORKSTATIONS', 'vni_id': 13100, 'addr': '10.199.14.1', 'mask': 24, 'mcast_grp': '239.0.0.14', 'vrf': 'Prod', 'tag': 54321})
TASK [dcb_vxlan : Configure VRF's under BGP] **************************************************************************************************************************************************************
changed: [wrlablfswb03] => (item={'vrf': 'Prod', 'vni_id': 13000, 'afi': 'ipv4', 'safi': 'unicast'})
ok: [wrlablfswb03] => (item={'vrf': 'Servers', 'vni_id': 23000, 'afi': 'ipv4', 'safi': 'unicast'})
TASK [dcb_vxlan : SAVE RUN CONFIG TO STARTUP CONFIG] ******************************************************************************************************************************************************
changed: [wrlablfswb03]
changed: [wrlabspswb01]
PLAY RECAP ****************************************************************************************************************************************************************************************************
wrlabbgwswb01 : ok=29 changed=4 unreachable=0 failed=1 skipped=3 rescued=0 ignored=0
wrlablfswb01 : ok=32 changed=4 unreachable=0 failed=1 skipped=3 rescued=0 ignored=0
wrlablfswb02 : ok=31 changed=3 unreachable=0 failed=1 skipped=4 rescued=0 ignored=0
wrlablfswb03 : ok=46 changed=7 unreachable=0 failed=0 skipped=5 rescued=0 ignored=0
wrlabspswb01 : ok=20 changed=3 unreachable=0 failed=0 skipped=8 rescued=0 ignored=0
Thanks for checking out my blog. If you've noticed anything missing or have any questions, please leave a comment and let me know.
Add new comment