Wireshark Cheat Sheet

Baseline Configuration

Edit > Preferences > Columns
Add column Title "ID", Type "Custom" & Fields "ip.id"
Add column Title "TCP Seq", Type "Custom" & Fields "tcp.seq"
Edit Time, set Type "UTC time"

Edit > Protocols > TCP
Untick Relative sequence numbers

View > Time Display Format > UTC Date and Time of Day

Quick Starting Points for Identifying Issues

Packet Loss
TCP Lost Segments (gaps in sequence numbers, indication of loss): tcp.analysis.lost_segment
Re-transmits (not necessarily packet loss) : (tcp.analysis.retransmission or tcp.analysis.fast_retransmission) and ip.addr==160.43.172.4
TCP Duplicate ACKs: tcp.analysis.duplicate_ack
TCP Resets: tcp.flags.reset==1 and ip.addr==160.43.172.4
TCP FIN: tcp.flags.fin==1 and ip.addr==160.43.172.4

Alternate syntax:
(ip.addr==10.83.230.68 && ip.addr==10.83.233.40) && (tcp.analysis.retransmission || tcp.analysis.fast_retransmission)

Proper verification of packet loss requires matching up packet captures from each end, re-transmits can be caused by CPU load, out of sequence packets, etc.
Absolute TCP sequence numbers, or packet IDs (for UDP) can be used to match packets at both ends.

DNS
DNS queries with no response:
dns and (dns.flags.response==0) and !dns.response_in

DNS queries that failed:
(!(dns.flags.rcode==0)) and (dns.flags.response==1)

Filtering
To exclude packets:
!(udp.port==3389)
!(ip.addr==10.83.157.11)

Note that != can have unexpected consequences

Decoding RTP Packets
If Wireshark could not auto-detect that UDP packets were RTP ones, you have to apply a display filter "udp and !rtp" and then randomly choose packets in the packet list, right-click them, choose Decode as... & mark them as RTP ones (both directions) until you have no packets left. Then you can remove the display filter and proceed to play back via Telephony > RTP > RTP Streams > Analyze > Play Streams.

Probably Overdue Update ;)

I haven't had any time for the blog due to real life commitments since about November 2022, but I have been plodding along with bug fixes & improvements to tooling, so head on over to my GitHub:

• Fixed the SNMP ping sweep to work with Aruba (+ other vendors), after wrangling with snmpwalk. These updates were then applied to the automated pre & post checks tool also.
  
• Improved the ACL decrufter via code simplification & bug fixes.
  
• Rewritten the switch MAC ARP DNS report to handle multiple switches, with ARP & DNS lookup details & Arista support also.
• Created a new tool switch MAC ARP DNS scraper, which attempts to answer the question "what's connected to what" for a given set of devices.
  

New Tool - Phone LSC Scraper

Over the years I've seen the CAPF Report in CUCM list incorrect certificate information quite a few times, which is awkward if you're using an LSC for VPN or .1x authentication & trying to report on incorrect or expired certificates.

The phone LSC scraper does a dynamic audit of certificates installed on phones by leveraging the AXL & RIS APIs. First it pulls list of SEP devices from AXL API, then uses this list to retrieve IP addresses of registered phones via the RIS API. Then it connects via HTTPS to each IP address & outputs the certificate subject & expiry date.

Configuration is taken from the same JSON files as the DN recording checker uses. However note that the application user requires Standard AXL API Access, Standard RealtimeAndTraceCollection & Standard Serviceability roles.

GitHub repo: https://github.com/Chris-P-15B/Voice-Automation

Example output:

python Phone_LSC_Scraper.py cucm-emea.json

Password:

160 SEP devices found in configuration.

SEP0004F2EBC0FE, 10.0.220.131, unable to connect.

SEP000832AA702F, 10.0.216.51, certificate subject {'serialNumber': 'PID:CP-8865 SN:FCH1136EABC', 'C': 'US', 'ST': 'NY', 'L': 'Albany', 'O': 'A Business', 'OU': 'IT Support', 'CN': 'CP-8865-SEP000832AA702F'}, expires 2026-10-07 11:14:06.

SEP000832AAAB7E, 10.0.216.134, certificate subject {'serialNumber': 'PID:CP-8865 SN:FCH1138DDEF', 'C': 'US', 'ST': 'NY', 'L': 'Albany', 'O': 'A Business', 'OU': 'IT Support', 'CN': 'CP-8865-SEP000832AAAB7E'}, expires 2026-10-07 11:14:09.

Speaking of the DN recording checker, that's been updated to include a column that describes the config issues found more clearly. It's also located in the Voice-Automation repo, along with instructions on creating the JSON configuration files.

New Tool - Automated Pre & Post Checks

Due diligence is dull! Capturing before & after outputs when performing changes, then running a diff to spot possible issues is time consuming. So I made a tool that's easily extendable to do the legwork for me.

It connects via SSH to a specified list of network devices, automatically detects the platform & runs platform specific commands. Features additional role specific checks based on partial hostnames, optional ping sweep (pulls interface IP addresses via SNMP) & VRF aware BGP peer routes check. HTML post checks report with command output diffs is emailed out to specified email address as a zip file attachment. Each SSH session to a device is handled in a separate thread, for reduced execution times when running against multiple devices.

The first run of the tool will create a directory in the temporary files path, named after the change control ID. The output of the pre-checks will be stored as text files in this directory.

The second run of the tool will store the outputs of the post-checks in this directory, run a diff against the pre & post checks, generate an HTML report & send an email with it attached as a zip file.

https://github.com/Chris-P-15B/Automated-Pre-and-Post-Checks

Snippet from an example checkout report:

New Tool - ACL Decrufter

I changed jobs last year, so have been rather busy learning all kinds of new stuff related to low-latency networking. Anyway limited TCAM capacity on low-latency Arista or Cisco Nexus switches makes for limitations on how much ACLs you can configure. Remediating badly written ACLs by hand is boring, so I made a tool...

https://github.com/Chris-P-15B/ACL-Decrufter

Parses IOS XE, NX-OS or EOS ACL output from show access-list command & attempts to de-cruft it by removing Access Control Entries (ACE) covered by an earlier deny, permit/deny with overlapping networks and/or merging permit/deny for adjacent networks.

Example output:

Original ACL:
deny tcp 172.30.0.0/24 172.31.0.0/24
deny udp 172.30.0.0/24 172.31.0.0/24 eq 443
permit udp 172.30.0.0/24 172.31.0.0/25
permit udp 172.30.0.0/24 172.31.0.0/25 eq 443
permit ip 172.16.0.0/23 10.1.1.1/32
permit udp 172.16.1.0/24 10.1.1.1/32
permit tcp 172.16.0.0/24 10.1.1.1/32
permit tcp 172.16.0.0/25 10.1.1.0/24
permit udp 192.168.0.0/24 192.168.1.0/24
permit tcp 192.168.0.0/24 192.168.1.0/24
permit ip 10.1.1.1/32 172.16.0.0/23
permit udp 10.1.1.1/32 172.16.1.0/24
permit tcp 10.1.1.1/32 172.16.0.0/24
permit tcp 10.1.1.0/24 172.16.0.0/25
permit udp 192.168.1.0/24 192.168.0.0/24
permit udp 192.168.1.0/24 192.168.0.0/25
permit udp 192.168.1.0/24 192.168.0.128/25
permit tcp 192.168.1.0/24 192.168.0.0/24
permit ip 172.16.0.0/22 10.1.1.1/32
permit ip 172.16.0.0/23 10.1.1.1/32
permit tcp 192.168.0.0/24 192.168.0.0/23
permit ip 172.20.0.0/24 any
permit ip 172.20.1.0/24 any
permit ip 172.20.2.0/24 any
permit ip 172.20.3.0/24 any
permit tcp 192.168.254.0/24 192.168.255.0/24 range 100 200
permit tcp 192.168.254.0/24 192.168.255.0/24 range 50 250
permit udp 192.168.254.0/24 192.168.255.0/24 range 50 250
permit udp 192.168.254.0/24 192.168.255.0/24 gt 40
permit udp 192.168.254.0/24 192.168.255.0/24 neq 39

Non-Overlapping Deny ACL:
deny tcp 172.30.0.0/24 172.31.0.0/24
deny udp 172.30.0.0/24 172.31.0.0/24 eq 443
permit udp 172.30.0.0/24 172.31.0.0/25
permit ip 172.16.0.0/23 10.1.1.1/32
permit udp 172.16.1.0/24 10.1.1.1/32
permit tcp 172.16.0.0/24 10.1.1.1/32
permit tcp 172.16.0.0/25 10.1.1.0/24
permit udp 192.168.0.0/24 192.168.1.0/24
permit tcp 192.168.0.0/24 192.168.1.0/24
permit ip 10.1.1.1/32 172.16.0.0/23
permit udp 10.1.1.1/32 172.16.1.0/24
permit tcp 10.1.1.1/32 172.16.0.0/24
permit tcp 10.1.1.0/24 172.16.0.0/25
permit udp 192.168.1.0/24 192.168.0.0/24
permit udp 192.168.1.0/24 192.168.0.0/25
permit udp 192.168.1.0/24 192.168.0.128/25
permit tcp 192.168.1.0/24 192.168.0.0/24
permit ip 172.16.0.0/22 10.1.1.1/32
permit ip 172.16.0.0/23 10.1.1.1/32
permit tcp 192.168.0.0/24 192.168.0.0/23
permit ip 172.20.0.0/24 any
permit ip 172.20.1.0/24 any
permit ip 172.20.2.0/24 any
permit ip 172.20.3.0/24 any
permit tcp 192.168.254.0/24 192.168.255.0/24 range 100 200
permit tcp 192.168.254.0/24 192.168.255.0/24 range 50 250
permit udp 192.168.254.0/24 192.168.255.0/24 range 50 250
permit udp 192.168.254.0/24 192.168.255.0/24 gt 40
permit udp 192.168.254.0/24 192.168.255.0/24 neq 39

Non-Overlapping Networks ACL:
deny tcp 172.30.0.0/24 172.31.0.0/24
deny udp 172.30.0.0/24 172.31.0.0/24 eq 443
permit udp 172.30.0.0/24 172.31.0.0/25
permit tcp 172.16.0.0/25 10.1.1.0/24
permit udp 192.168.0.0/24 192.168.1.0/24
permit ip 10.1.1.1/32 172.16.0.0/23
permit tcp 10.1.1.0/24 172.16.0.0/25
permit udp 192.168.1.0/24 192.168.0.0/24
permit tcp 192.168.1.0/24 192.168.0.0/24
permit ip 172.16.0.0/22 10.1.1.1/32
permit tcp 192.168.0.0/24 192.168.0.0/23
permit ip 172.20.0.0/24 any
permit ip 172.20.1.0/24 any
permit ip 172.20.2.0/24 any
permit ip 172.20.3.0/24 any
permit tcp 192.168.254.0/24 192.168.255.0/24 range 50 250
permit udp 192.168.254.0/24 192.168.255.0/24 neq 39

Merged Adjacent Networks ACL:
deny tcp 172.30.0.0/24 172.31.0.0/24
deny udp 172.30.0.0/24 172.31.0.0/24 eq 443
permit udp 172.30.0.0/24 172.31.0.0/25
permit tcp 172.16.0.0/25 10.1.1.0/24
permit udp 192.168.0.0/24 192.168.1.0/24
permit ip 10.1.1.1/32 172.16.0.0/23
permit tcp 10.1.1.0/24 172.16.0.0/25
permit udp 192.168.1.0/24 192.168.0.0/24
permit tcp 192.168.1.0/24 192.168.0.0/24
permit ip 172.16.0.0/22 10.1.1.1/32
permit tcp 192.168.0.0/24 192.168.0.0/23
permit ip 172.20.0.0/22 any
permit tcp 192.168.254.0/24 192.168.255.0/24 range 50 250
permit udp 192.168.254.0/24 192.168.255.0/24 neq 39

Decrufted ACL:
deny tcp 172.30.0.0/24 172.31.0.0/24
deny udp 172.30.0.0/24 172.31.0.0/24 eq 443
permit udp 172.30.0.0/24 172.31.0.0/25
permit tcp 172.16.0.0/25 10.1.1.0/24
permit udp 192.168.0.0/24 192.168.1.0/24
permit ip 10.1.1.1/32 172.16.0.0/23
permit tcp 10.1.1.0/24 172.16.0.0/25
permit udp 192.168.1.0/24 192.168.0.0/24
permit tcp 192.168.1.0/24 192.168.0.0/24
permit ip 172.16.0.0/22 10.1.1.1/32
permit tcp 192.168.0.0/24 192.168.0.0/23
permit ip 172.20.0.0/22 any
permit tcp 192.168.254.0/24 192.168.255.0/24 range 50 250
permit udp 192.168.254.0/24 192.168.255.0/24 neq 39

Incident Models

An incident model is a means to streamline & standardise the troubleshooting of critical business systems. As a guide it should include the following information:

• Overview of the system or application
• Topology diagram(s) and/or list of devices
• Flowchart for the incident handling process, with checkpoints and/or milestones
• Template for notifications to the business or stakeholders
• Basic troubleshooting commands
• Links to support contracts & contacts

Additional information can be included, such as links to in depth troubleshooting guides, the original design documentation or configuraton backups. Sometimes I like to include keywords that may be mentioned in tickets related to this system, to help 1st line staff quickly triage an incident & apply the appropriate incident model.

Cisco Switch MAC Address Flapping Alerts

MAC address table instability can impact a switch's performance & on lower end switches cause high CPU utilisation that may impact other functions. Cisco switches can generate a syslog entry when they see a MAC address flap between ports, but it’s not enabled by default. Some NX-OS platforms actually temporarily disable MAC address table updates if a certain number of MAC address flaps occur within a set timeframe: https://www.cisco.com/c/en/us/support/docs/ios-nx-os-software/nx-os-software/213906-nexus-9000-mac-move-troubleshooting-and.html
The different switch platforms generate slightly different syslog messages, but the common factor is they all have MAC_MOVE in the text for NX-OS, or MACFLAP or HOSTFLAP for IOS / IOS XE. So I created an alert in Splunk to match these keywords in the last hour's log entries.

Commands

IOS / IOX XE:
mac address table notification mac-move

N3K:
mac address table notification mac-move
logging level fwm 6
logging monitor 6

N4K:
mac address table notification mac-move
logging level fwm 6
logging monitor 6

N5K / N6K:
mac address table notification mac-move
logging level fwm 6
logging monitor 6

N7K / N9K:
logging level l2fm 5