Cisco ISR4K Throughput related hidden command

If you have ever worked with Cisco ISR4K platform, you probably know that these routers have plenty of horsepower. That is, you will probably hit the licensed throughput limit before you even get to 50% on CPU. This can be very frustrating when you troubleshoot, because you need to know (a) platform’s throughput (b) current load. What I always hated is the fact that Cisco doesn’t make required information available to public. For example, the only document about ISR4K performance I found was classified as ‘Cisco Confidential’ and even that one lacked any information about troubleshooting guidelines. Anyway, I’ll keep this post short. Here’s the troubleshooting methodology I use (inc hidden commands I got from Cisco TAC). Read more …

ECSE Takeaways

Have just finished Ekahau Site Survey course in Oxford and here are my takeaways from it:

  • Move away from 2.4GHz completely (inc. BYOD and Guest networks if possible) – don’t use it in modern world
  • Avoid using multiple SSIDs – beacons make the air dirty (have captured 60MB of beacons on one channel in 15m as a test)
  • Don’t install APs in corridors or hallways – signal from omni antennas propagates on hundreds of meters in free space
  • Make sure signal strength from 2nd strongest APs meet the main requirements if voice roaming is of concern. This is an equivalent of Cisco’s channel overlap of 10-20-30% requirement, which is hard to measure or prove.
  • If APs are installed in rooms/offices, put them away from windows to avoid waste of RF energy sent towards the street.
  • Mirrors do not significantly affect 2.4GHz (and 5GHz) – waves are still lengthy enough to penetrate through mirrors (?).
  • Adjacent channels interference is more destructive than co-channel interference, because devices on adjacent channels do not play nicely when they access the media (do not contend)
  • Radio Tap header information is added by the client. RSSI and noise levels reported by clients are virtual and vendor specific. NICs do not see RF shapes to measure noise and RSSI, but spectrum analyzers do.
  • Ch144 on 5GHz was introduced with 802.11ac
  • ESS cheats
    • Don’t use small objects to define scale, such as doors, desks and so on. This increases error level.
    • Use distance between objects to define scale on one floor, then use alignment points to define scale on all others.
    • Use coverage and coverage exclusion areas to make ESS reporting more accurate (coverage percentage)
    • If floors are more or less identical, draw one and duplicate the rest (using image swap trick). This will help to save a lot of time as this trick copies all objects, including alignment points and scales.
  • Useful Tools
    • Buffalo WMR-433: Very tiny WiFi router which can reach out on tens of meters
    • Theodolite: iOS app to measure azimuth, elevation, GPS coordinates – helps with outdoor deployments
    • NetScout Link Sprinter 300: A link tester. Checks all layers of TCP/IP protocols suite (physical/data link, network, transport and application). Capable to send results by email.
    • HORST: linux-based lightweight 802.11 analyzer with text interface
    • Kismet: linux-based wireless sniffer/packet capture software

That was a great week in Oxford. Hopefully I haven’t misinterpreted the information above ūüôā

Directed Broadcasts and IOS

I’ve been reading about directed broadcasts recently. According to RFC2644, directed broadcasts should not be forwarded by routers to the end hosts on the destination subnet. It wasn’t very clear if routers should drop directed broadcasts. If so, should these be dropped by intermediate router, or the one at the final destination? So, I decided to build a small lab and capture few packets. Read more …

Cisco WLC SFP Inventory

We all know about very useful IOS command show inventory, which displays asset’s detailed information, including installed SFPs and their serial numbers. However, in case of AireOS, show inventory supplies very limited information – only chassis model and serial number, there’s no information about SFPs. Well, is there a way to find SFPs models and serial numbers? Yes, with the help of the following hidden AireOS command:

debug fastpath cfgtool --dump.sfp

Here’s an example output from one of ours WLCs:

(Cisco Controller)>FP0.
Port SFP Vendor       Transceiver Type    OUI PartNumber       Rev  SerialNumber     DateCode Auth
   1 CISCO-METHODE    (0x08)1000BaseTX        SP7041-R              MTC201XXXXX      16030701 ok
   2 CISCO-METHODE    (0x08)1000BaseTX        SP7041-R              MTC201XXXXX      16030701 ok

The model, the part and serial numbers are all there.

I hope you find this helpful.

Convert Lightweight AP into Autonomous AP

There are a number of ways to convert Lightweight AP into Autonomous. I personally prefer two.

First method is to use Console connection. It works very well, but of course requires a console, which in some cases can be troublesome¬†– i.e. someone¬†will have to arrange a PC connected to the network and console connection to the AP, which also has to be reachable from the PC once IP address is configured (i.e. L3 reachability). It also requires to press and hold Mode button before powering up an access point, which then has to be held up to 30 seconds until LED turns red. After that Mode button can be released and AP falls into rommon. Now, it’s just a matter of few commands:

tar -xtract tftp:// flash:

These commands assign IP address, netmask and default gateway to AP, initialize TFTP service and then download .tar archive with the image from TFTP server ( and extract its contents into flash. After this step is completed, AP can be either power cycled, or forced to boot with new image (using “boot” command). Also, I recommend to format flash before extracting new image ¬†using “format flash:“.

Works like a charm, but as I previously stated, requires a direct console connection.

Second method allows to downgrade Lightweight AP with Autonomous IOS without console access, but it requires AP to be accessible via SSH or Telnet. Once in AP’s CLI execute the following:

debug capwap console cli

This command enables full range of CLI commands, like config terminal, archive, or format flash (which by default are not available in LAP’s CLI). Therefore, it is now possible to do the following

archive download-sw /overwrite tftp://

Reboot and you’re on IOS code now.

Hope this helps.


Cisco Prime: Catalyst 3850 image archiving

It’s been a while since I’ve posted anything on my website. I had a crazy year, had a lot of work that ruined all my plans to study a lot. Hopefully it’s now sorted with my management and next year is promised to be much better from the work/life/study balance perspective. I know I wasn’t a best webmaster and didn’t respond to many of your comments – I promise I will improve and reply to all comments in the coming weeks. I am going for WISECURE exam in less than two weeks from now, so wish me luck.

In the meanwhile, I’d like to share an undocumented feature of Cisco Prime (well, at least I wasn’t able to find any guidelines on that in Cisco Prime documentation). I have recently configured Prime for images archiving, but few days later realized that it fails to copy images from Catalyst 3850 switches. All transfer modes have been failing – FTP, SFTP, SCP and TFTP. There are no settings in Prime that relate to SCP and SFTP (you can only enable/disable FTP and TFTP servers). I’ve done some research and found that for SCP downloads Prime is acting as a client. Therefore, Catalyst 3850/3650 switches must be SCP servers.

To enable SCP server on Catalyst 3850/3650 execute the following global configuration command:

ip scp server enable

Once applied, Prime will be able to archive IOS-XE images from Catalyst 3850/3650s. However, there is restriction. Cisco Prime cannot archive an image from Catalyst 3850 if it was installed using INSTALL mode. That is, when BIN package is unpacked into few separate files. Only BUNDLE mode is supported for archiving. Not great huh? Cisco does not recommend to use BUNDLE mode for image distribution on Catalyst 3850s, but at the same time Cisco Prime can’t archive image if it was distributed using recommended INSTALL mode.

Cisco is so Cisco…

Cisco Prime: Radiation Patterns and Antennas Orientation

Sorry for being silent for a while. I have moved to Leeds and now work in my company’s HQ as a project and design engineer. A massive change for me and my family. Anyway, this time I want to cover one very important topic – radiation patterns and antennas orientation in Cisco Prime (also applies to old buddy WCS).

If you have ever worked with Cisco Prime (or WCS) with regards to wireless networks management, you know how frustrating can be the process of adjusting access points’ radiation characteristics on the floor plan. Actually, it’s not a big deal if all access points have internal antennas and have been installed as per¬†Cisco’s recommendations. In such case, Cisco Prime applies default azimuth and elevation values to match best practice installation.¬†For example, omni-directional APs, like 1142N, 3602i or 3702i, have internal antennas and provide best coverage if installed¬†on the ceiling, facing down.¬†Of course, they can still provide optimal coverage in some scenarios with¬†wall mount installations, but in such cases a more proper planning is required. There is a high likelihood that access points from different floors will become adjacent in RF spectrum (i.e. will see each other) and it will be more complex for a controller to come up with optimal Tx power levels to meet coverage requirements. However, we all know that suboptimal installations happen in a real life. In such cases every access point has to be configured with custom azimuth and elevation values to help Cisco Prime to build correct heatmaps. This is especially important during the planning phase.

Note! In case of external antennas (i.e 3702e with patch or sectoral antenna), it is always required¬†to adjust elevation and azimuth within Cisco Prime to let it know how antennas radiate on the floor plan – north, south, west, east, north-west, south-east. This process can be even more complicated if antennas are installed at 45 (or custom) degrees to the floor/wall/ceiling. Read more …

Catalyst 3850 QoS Troubleshooting Commands

You would expect to have a very simple and intuitive QoS troubleshooting toolkit on a native MQC platform. Well, the first thing that came into my mind after I’ve been told that Catalyst 3850 is a proper MQC platform was something like

Wow, finally I can use the very same range of ISR commands to troubleshoot QoS on a hardware switching platform!

Unfortunately, that ended up to be a hybrid implementation. Yes, we have to use MQC to configure QoS on Catalyst 3850, including queuing! No, you are unlikely to solely use MQC commands to troubleshoot QoS. Remember that show mls qos commands range on Catalyst 3750 platform? It’s still the same candy, in a different wrapping though. Read more …

Catalyst 3850 Flexible NetFlow restrictions

Cisco Catalyst 3850 has become a next generation switching platform in our company. We have deployed these switches in a number of our offices recently. Apart from being a converged wired/wireless access platform, it fully supports Flexible NetFlow. Therefore, it was a logical step to begin using this neat feature at least on our branche core switches to improve monitoring capabilities and lessen troubleshooting efforts. I have faced a number of issues while I’ve been trying to configure FNF on the first switch to perform some testing. So… Read more …