Cisco Virtual Stackwise with Catalyst 9k switches
Recently I've been tasked with replacing some old switch stacks at work with some new shiny Catalyst 9500 switches and started looking into how to Stack these switches as they no longer support stacking modules which at first seemed odd to me given that the backplane for Stackwise was like 160Gbps for the Cataylst 3600/3800 series switches from memory but it seems Cisco have decided to move to a Virtual Stackwise feature which uses the SFP/SFP+/QSFP ports on the switch instead of the dedicated Stacking Ports at the back. This makes sense really given then these days running 100Gbps fibre isn't uncommon in a modern network and also allows you to run much longer fibre leads between switches to stack them in separate racks which again, design wise can make a lot of sense. Hell, you could stack switches across a campus if you really wanted to which while might not be best practice, or practical, it's pretty damn cool.
What is StackWise Virtual?
StackWise Virtual operates in pretty much the exact same was as traditional StackWise does using stack modules with the caviat that you are limited to two physical switches whereas traditional Stackwise using stack modules allows for up to 8 switches to be stacked and appear as a single Virtual Switch. Even still, using Virtual Stackwise, still allows for enhanced network design in regard to scalability, high availability, and redundancy. It also means less physical devices to configure as multiple switches are all configured from a single management plane and share the same control plane as well so long as stacking 2 switches is all you need. There are some advantages to virtual stacking like this as well, as you are not limited to the length of a stack cable (3M i believe is the longest). Instead, you can run it as long as you can run a fibre lead. This does allow for some flexibility in stacking switches where you can stack two switches that are in completely different data halls let alone completely different racks.
Virtual Stackwise uses Stateful Switchover (SSO) as well as Non-Stop Forwarding (NSF) to maintain routing protocols and provide traffic failover in an almost seemles way. When I say almost, I mean you may notice a tiny blip but that will mean nothing to your production data. When you create a Virtual Switch stack, you must configure a StackWise Virtual Domain. This Virtual domain will have exactly one Active switch, and one Standby switch. Within a Stackwise Virtual domain, all control plane functions are centrally managed by the Active switch. This includes, SNMP, Telnet, SSH, BPDU's, LACP, Routine protocols etc.
Withing the Stackwise domain switches, upon boot, the Active node will assign a virtual MAC address to all of it's layer 3 Interfaces. This MAC is derived from an EEPROM memory device located on the Switch. The Active switch will then provide this MAC address as the MAC for that Virtual Domain. The standby switch will also select a Virtual MAC derived from the same process, however in the event of a failover. The Standby devices virtual MAC, is not used. The Standby device in the event of a failover, will start using the MAC provided by the Active device and the Active device will take over the Standby devices Virtual MAC.
As I mentioned earlier, in order to connect to two switches together, you need a Virtual Link. This virtual link is a special interface type that forms an Etherchannel interface between the two switches and can comprise of up to eight physical interfaces. The virtual link will carry two types of traffic, Control Traffic, and normal data traffic. The Control traffic that goes across these Virtual Links is prioritised over Data traffic to ensure that they get sent via priority queues on the egress interfaces. Both control and data traffic however get encapsulated inside a StackWise Virtual header. This ensures that the frame information is conveyed correctly on both switches withing the virtual stack. Let's take a look at how to configure two switches in a StackWise Virtual configuration.
Configuring StackWise Virtual
To begin, you need to make sure you are running a version of IOS-XE that supports Vritual Stackwise which is 16.8.1 for the Catalyst 3650XS and all 9500 series switches, 16.9.1 for catalyst 9400 series switches and 16.12.1 for Catalyst 9606 switches (Quad sup support requires 17.2.1) Once you're on the correct IOS-XE version, you will also now need to make sure you have the right license level. For Virtual StackWise, you now need network-advantage licensing. To enable the correct license level, run the following command on both switches and reboot.
Switch(config)#license boot level network-advantage addon dna-advantage
% use 'write' command to make license boot config take effect on next boot
Switch(config)#end
Switch#wr
Building configuration...
*Mar 15 01:55:13.544: %SYS-5-CONFIG_I: Configured from console by console[OK]
Switch#Once the switch has booted with the correct license level, you can enable stackwise-virtual on both switches. Configure the StackWise virtual domain, save the changes and once again reboot both switches.
Switch#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Switch(config)#stackwise-virtual
Please reload the switch for Stackwise Virtual configuration to take effect
Upon reboot, the config will be part of running config but not part of start up config.
Switch(config-stackwise-virtual)#domain 1
Switch(config-stackwise-virtual)#end
Switch#wr
Building configuration...
*Mar 15 01:59:49.350: %SYS-5-CONFIG_I: Configured from console by console[OK]
Switch#reload
*Mar 15 01:59:59.005: %SYS-6-PRIVCFG_ENCRYPT_SUCCESS: Successfully encrypted private config file
Proceed with reload? [confirm]Once the StackWise-Virtual domain has been configured, and the switches come back up, it's time to configure the Stack interfaces and the DAD (Dual Active Detection) interface/s. This needs to be done on both switches.
Switch#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Switch(config)#interface TwentyFiveGigE1/0/48
Switch(config-if)# stackwise-virtual dual-active-detection
WARNING: All the extraneous configurations will be removed for TwentyFiveGigE1/0/48 on reboot.
INFO: Upon reboot, the config will be part of running config but not part of start up config.
Switch(config-if)# description Virtual Stack Heartbeat
Switch(config-if)#
Switch(config-if)#interface HundredGigE1/0/51
Switch(config-if)# stackwise-virtual link 1
WARNING: SVL configuration will be ignored on lower (1G) speed.
WARNING: All the extraneous configurations will be removed for HundredGigE1/0/51 on reboot
INFO: Upon reboot, the config will be part of running config but not part of start up config.
Switch(config-if)# description Virtual Stack port
Switch(config-if)#
Switch(config-if)#interface HundredGigE1/0/52
Switch(config-if)# stackwise-virtual link 1
WARNING: SVL configuration will be ignored on lower (1G) speed.
WARNING: All the extraneous configurations will be removed for HundredGigE1/0/52 on reboot
INFO: Upon reboot, the config will be part of running config but not part of start up config.
Switch(config-if)# description Virtual Stack port
Switch(config-if)#
*Mar 15 02:04:05.604: %LINK-3-UPDOWN: Interface HundredGigE1/0/51, changed state to up
*Mar 15 02:04:05.750: %LINK-3-UPDOWN: Interface HundredGigE1/0/52, changed state to up
*Mar 15 02:04:06.605: %LINEPROTO-5-UPDOWN: Line protocol on Interface HundredGigE1/0/51, changed state to up
Switch(config-if)#
*Mar 15 02:04:06.665: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to up
*Mar 15 02:04:06.750: %LINEPROTO-5-UPDOWN: Line protocol on Interface HundredGigE1/0/52, changed state to up
Switch(config-if)#Once you have your interfaces configured, save the changes and reboot the switches again. If you're on the console, eventually you will see a message about joining the stack and the switch rebooting again.
Waiting for remote chassis to join
##################################################
Chassis number is 2
All chassis in the stack have been discovered. Accelerating discovery
Chassis is reloading, reason: Configured Switch num conflicts with peer, Changing local switch number to 2 and reloading to take effect
Mar 15 02:09:57.776: %PMAN-3-PROCHOLDDOWN: R0/0: pman: The process nif_mgr has been helddown (rc 69)
Chassis 2 reloading, reason - Reload commandAnd that's it, the switches are now configured in a Virtual Stack and will appear as a single logical switch.
Verifying StackWise Virtual
To verify the switch stacking you can use the command show switch. This displays a brief output of the switches that are configured in the Virtual stack and their status.
Switch#sh swi
Switch/Stack Mac Address : bc8d.1f6c.6100 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
-------------------------------------------------------------------------------------
*1 Active bc8d.1f6c.6240 15 V02 Ready
2 Standby bc8d.1f6c.6100 1 V02 ReadyAs you can see, both switches are up with switch 1 being the Active and 2 being the Standby. (I did have to renumber these switches after they joined the stack as they joined in the opposite order to what I wanted. To do that you use the command switch X renumber X on both switches, save the changes and reboot. It's that easy. You can also use the priority command as I did on switch 1 to ensure it's the Active switch. The command to set the priority is switch X priorit X with a priority value of 1-15 with 15 being the most preferred)
If you want more information regarding the StackWise Virtual ports etc, there are various show commands available. Below I have list the output of a few just to give you an idea.
Switch#sh stackwise-virtual
Stackwise Virtual Configuration:
--------------------------------
Stackwise Virtual : Enabled
Domain Number : 1
Switch Stackwise Virtual Link Ports
------ ---------------------- ------
1 1 HundredGigE1/0/51
HundredGigE1/0/52
2 1 HundredGigE2/0/51
HundredGigE2/0/52
Switch#sh stackwise-virtual dual-active-detection
In dual-active recovery mode: No
Recovery Reload: Enabled
Dual-Active-Detection Configuration:
-------------------------------------
Switch Dad port Status
------ ------------ ---------
1 TwentyFiveGigE1/0/48 up
2 TwentyFiveGigE2/0/48 up
Switch#sh stackwise-virtual bandwidth
Switch Bandwidth
------ ---------
1 200G
2 200G
Switch#sh stackwise-virtual switch 1
Stackwise Virtual Configuration:
--------------------------------
Stackwise Virtual : Enabled
Domain Number : 1
Switch Stackwise Virtual Link Ports
------ ---------------------- ------
1 1 HundredGigE1/0/51
HundredGigE1/0/52
Switch#sh stackwise-virtual switch 2
Stackwise Virtual Configuration:
--------------------------------
Stackwise Virtual : Enabled
Domain Number : 1
Switch Stackwise Virtual Link Ports
------ ---------------------- ------
2 1 HundredGigE2/0/51
HundredGigE2/0/52
That's it for todays StackWise Virtual post. If you've noticed anything missing of have any issues with this post, please leave a commend and let me know.
Add new comment