Network recording for error finding

Sometimes, you can have occasional network errors, like stream disconnecting, without having a clue of why.

A good help is to record the network exchanges continuously, so you can get back the network capture afterwards to analyze it.

This can be done by using Wireshark, or its command-line version tcpdump under Linux or Windows.

Wireshark

To use Wireshark, you need to have a user logged in and a graphical desktop interface.

1. Launch Wireshark
2. Go into Capture->Options
3. In the Input Tab:
   1. Select the network interface
   2. Optionally set a filter
      1. To select conversation with one host IP, set in the capture filter : host 192.168.1.12
      2. To select only TCP protocol set tcp or to select udp set udp
      3. To select only conversation for one port set port 80
      4. To select only TCP push packets (tcp and (tcp[tcpflags] & (tcp-push) != 0)
      5. You can combine all those with and, or, not and use parenthesis ( ).
      6. Example: to get all HTTP traffic with 173.236.178.205: host 173.236.178.205 and port 80 and (tcp and (tcp[tcpflags] & (tcp-push) != 0))
4. Go to the Output Tab and
   1. Select a base file name for storage
   2. Select pcapng format
   3. Check "Create a new file automatically"
   4. Check "When time is a multiple of" and choose for instance 15 minutes.
   5. Check "Use a ring buffer with" and select enough files to cover the time you need from detection to handling. To cover 24 hours with 15 minutes files, you need 24*4=96 files.
5. Go to the Options Tab:
   1. Uncheck "Update list of packets in real-time" to avoid memory increasing too much if you let it run for days

Depending on what you record and how big the traffic is, you should select a proper time range for each file.

When you detect a problem, you can then go to the storage folder of files, and copy the one with match the time when the problem happened, so it will not be deleted in next ring buffer.

Note : Wireshark needs special privileges to capture Ethernet packets. See https://wiki.wireshark.org/CaptureSetup/CapturePrivileges

Multicast on Windows:

You need an external program to join the multicast. You may use mcasttest.

If you need to select a specific interface, you need to add a route : route add -p 239.192.13.1 mask 255.255.255.255 192.168.5.27 metric 3 if 14

Use route print to find if number from MAC address.

Tcpdump

This works exactly as Wireshark, but in command-line, so you can run it in a Linux screen virtual TTY or a with nohup tcpdump ... &.

For instance:

tcpdump -i en0 -w /var/tmp/capture-%m-%d-%H-%M-%S-%s.pcapng -W 96 -G 900 "host 192.168.1.12 and tcp and port 80"

will capture from interface en0 filtering all HTTP traffic with 192.168.1.2, and save it to files in /var/tmp/capture-....pcapng (filling the name with date/time) every 900 seconds (so 15 minutes), and limited to 96 files (so 1 day = 24*4).

Manual can be found here.

Standalone

You do not have access to the OS in the Standalone processors, but you can use a switch port mirroring to record all conversation.

1. Configure the switch to mirror the port of the Standalone processor you want to monitor
2. Connect your recording PC to the mirroring port
3. Use Wireshark or Tcpdump to record the traffic.

Analyzing

There is no generic method to analyze what when wrong on a network, but here are some ideas.

Checking TCP acknowledge time

When there is a problem on the line for TCP communication, the ACK may be delayed a lot, creating errors for software with timeouts.

To see ACK arriving with more than 500ms, use filter tcp.analysis.ack_rtt >= 0.5

Checking TCP end of sessions

Use filter tcp.connection.fin_active