Choosing between Dynamic and Static FEX interfaces pinning

Cisco Nexus 2200 Fabric Extenders can be connected to the parent switches using two different modes: Static and Dynamic interfaces pinning. Static pinning mode instructs the switch to virtually split FEX into few blocks of ports and statically associate each block of ports with its own physical uplink. In other words, if one particular uplink fails, a range of FEX ports, associated with this uplink, fail as well. Hence, the word Static. Dynamic pinning mode is based on a Port-Channel logic. In very basic scenario you would have all your physical uplinks associated with a single Port-Channel that will stay up as long as there is at least one working physical uplink. At first glance, difference seems to be obvious and not in the favor of Static mode. But let’s dive into the subject to understand when Static mode becomes handy.

First, take a look at the following diagram that visualize the logic behind both methods.

Static vs Dynamic FEX Pinning

Both methods are interesting and, what is important, useful in different situations. One of the main drivers to consider the choice of the Static pinning mode is over-subscription. Imagine a Dual-Homed FEX design (reviewed previously in detail) with two uplinks per Nexus switch and Active/Standby NIC teaming at servers’ end. For the sake of clarity let’s agree that Dynamic pinning method is being used. Thus, two available uplinks are organized into Port-Channel providing 20Gbps throughput from the FEX to every N5K switch of a vPC pair. Well, virtual 20Gbps throughput, we won’t touch Port-Channel logic this time. Only one pair of N2K-N5K will be used by any server at any given time due to its Active/Standby nature. The other server’s NIC will be inactive until failure happens.

So, 20Gbps of throughput is shared among 48 x 1Gbps ports what results into 2.4 to 1 over-subscription ratio!  Imagine now that one of the uplinks fails. This doesn’t bring Port-Channel down (we still have one live uplink!), as such FEX continues to work normally as there were no issues at all. In result, over-subscription ratio jumps up to undesirable 4.8 to 1 ratio.

To avoid bottlenecks consider to use Static pinning mode. Imagine same topology but instead of Dynamic, Static pinning mode with two uplinks has been configured. Same initial over-subscription rule applies – 2.4 to 1, but due to the Static mode nature it won’t change no matter what happens. With only two uplinks from N2K to N5K, Nexus switch will virtually split FEX into two parts, each associated with its own physical link. 48ports / 2 = 24 ports per uplink, hence the same initial over-subscription ratio.

If one N2K-N5K uplink fails, all 24 associated FEX ports will fail as well, forcing the server to swap Active/Standby roles of its NICs and use an alternative FEX. In result, we still have High Availability, but we also preserve over-subscription ratio during failover scenario. Isn’t it good?

Again, it is only one particular case – Dual-Homed FEX with Active/Standby NIC teaming. While some scenarios may force you to use Dynamic pinning mode, the others won’t give you a desirable behavior without using the Static pinning mode. Take a look at the following list of possible scenarios with my brief comments.

  • Single-Homed FEX (w/o vPC) and Active/- servers – Dynamic mode provides High Availability from the uplink failure perspective.
  • Single-Homed FEX (w/o vPC) and Active/Passive servers – Assuming NICs are plugged into different port groups, static mode provides High Availability and, if LACP port-channel is used to connect the server with two NICs, then over-subscription ratio is halved (until failure happens);
  • Single-Homed FEX and Active/Passive servers – With all four uplinks, Static mode can provide High Availability and 1.2 to 1 over-subscription ratio (for single NIC of course);
  • Dual-Homed FEX and Active/Active servers (Enhanced vPC) – Both methods provide High Availability due to the resilient nature of the Enhanced vPC feature. Port-Channel hashing algorithm may chose sub-optimal paths though, if Dynamic pinning mode is being used and there’s at least one failed uplink from FEX to Nexus switch;

Hope this helps!

3 Comments

  1. MJ says:

    Hello Tim,

    does using port-channel implies dynamic pinning or does it have specific configuration?

    • Hi MJ!

      It was long time ago, when I last configured a Nexus switch. If I still remember it right and If I understand your question correctly, port-channels can be used in hybrid configurations as well… Static pinning implies you configure your fex with “pinning max-links X” and use X amount of physical interfaces. However, you can pin 2 links, but instead of physical ports, you can still use port channels, e.g. create two port-channels with two ports each and pin both port channels to the specific fex. This gives you a hybrid config, where half of fex’s ports will use one port channel (and its physical members), while another half will use another port channel. If one link fails it will only affect over subscription of one half of fex’s ports…

      However, dynamic pinning means you need to use a port channel. Hence the name… if you don’t use them (use physical ports) then some amount of ports on a fex will fail with the failure of physical link.

Leave a Reply

%d bloggers like this: