Export raw packet bytes in tshark, tcpdump, or similar?

Question

Context:

I have a *.pcap file with many WLAN probe requests. My goal is to extract the WLAN management frame of each probe request as raw bytes (that is, no headers and no extra information - only the raw bytes like they were originally captured).

In Wireshark, I can simply right-click the management frame and select "Export Packet Bytes...":

If I select "RAW" as the file format, Wireshark gives me exactly what I want: A file containing nothing more than the selected bytes.

Problem:

I need to automate this task programatically. I came across tools like tshark, tcpdump, capedit, etc. However, none of these tools seem to allow me to extract the WLAN management frame and nothing more.

While I was able to get the desired bytes as ASCII on stdout using tcpdump, I wasn't able to save them to a file or into a variable. Additionally, I was only able to do it for a single probe request, not for all probe requests in the *.pcap file.

Current Approach:

As mentioned above, I was able to get the desired bytes as ASCII on stdout:

$ tcpdump -r capture.pcap -c 1 -x
reading from file capture.pcap, link-type IEEE802_11_RADIO (802.11 plus radiotap header)
11:24:52.933799 1.0 Mb/s 2457 MHz 11b -77dBm signal antenna 1 Probe Request () [1.0* 2.0* 5.5* 6.0 9.0 11.0* 12.0 18.0 Mbit]
    0x0000:  0000 0108 8284 8b0c 1296 1824 3204 3048
    0x0010:  606c 2d1a ac01 02ff ff00 0000 0000 0000
    0x0020:  0000 0000 0000 0000 0000 0000 0000

To extract the raw bytes, I can simply pipe this output into grep, sed, and xxd:

$ tcpdump -r capture.pcap -c 1 -x | grep "0x00" | sed  's/0x[[:xdigit:]]*:[[:space:]]*//g' | xxd -r -p > rawbytefile

Obviously, this is a rather hacky way of accomplishing what I want and there must be a better way. Nobody wants code that's highly dependent on the human-centric output of other people's programs.

Questions:

1. How can I extract the WLAN management frame in a proper way? (On the command line/programatically - using Bash, Python, etc.)
2. How can this be done for every probe request within the *.pcap-file? (Again, in a proper way - building another hacky loop by parsing ASCII output is not really what one should do...)

Micky

Answer 1

I suppose you've found a solution already, but I thought I would post a possible alternate solution that may or may not meet your needs and which utilizes only tshark. If you disable the "wlan_mgt" protocol, the "wlan" payload gets passed to the generic "data" dissector, which can be printed. For example:

tshark -r capture.pcap --disable-protocol wlan_mgt -Y "wlan.fc.type_subtype == 0x0004" -T fields -e data

... or with colon-separators between the bytes if you prefer:

tshark -r capture.pcap --disable-protocol wlan_mgt -Y "wlan.fc.type_subtype == 0x0004" -T fields -e data.data

Older versions of tshark didn't allow you to disable a specific protocol on the command-line, so in that case you would have to disable the "wlan_mgt" dissector in Wireshark first or create a separate Wireshark profile in which it was disabled and then tell tshark to use that profile via the -C <configuration profile> option.

I'm running an older version of Wireshark (1.12.13) and in my testing there appears to be a bug because only a subset of the probe request packets were printed. Perhaps the bug is fixed in newer versions of Wireshark though, but if not, a Wireshark bug report could be filed requesting for this problem to be fixed.

Lys

Answer 2

You can achieve it using my Java library, Pcap4J.

Add pcap4j-core.jar and pcap4j-packetfactory-static.jar to your class path and execute code like below:

PcapHandle handle = Pcaps.openOffline("/path/to/your.pcap");
while (true) {
  try {
    Packet packet = handle.getNextPacketEx();
    Dot11ProbeRequestPacket probe = packet.get(Dot11ProbeRequestPacket.class);
    if (probe == null) { continue; }
    byte[] mgmtFrame = ByteArrays.getSubArray(
      probe.getRawData(),
      probe.getHeader().getFrameControl().isOrder() ? 28 : 24
    );
    System.out.println(ByteArrays.toHexString(mgmtFrame, ""));
  } catch (EOFException e) {
    break;
  } catch (PcapNativeException | TimeoutException | NotOpenException e) {
    e.printStackTrace();
  }
}
handle.close();

Depolymerize

Thank you! Unfortunately, your answer didn't really fit my particular use case (Java is too heavy in this case). But you definitely answered my initial questions and led me into the right direction. See my answer below for a more lightweight solution. – Micky

Answer 3

Similar to kaitoy's answer, but using the scapy library for Python:

from scapy.all import *

with PcapReader('/root/capture.pcap') as pcap_reader:
    for pkt in pcap_reader:
        if pkt.haslayer(Dot11ProbeReq):
            probe_req = pkt.getlayer(Dot11ProbeReq)
            raw_probe_req = bytes(probe_req)
            hexdump(raw_probe_req)

Micky

While it's not tshark like the original request, it is scapy so this actually plugs in really nicely to a lot of automated test systems - thanks! – Marenmarena

Answer 4

The following example pulls binary dump from specific field out of all packets filtered. Replace the "nfs.data" field name if you need to pull from some other field. To quickly get correct field name - open WireShark, expand packet to location you want, right-click on field --> Copy --> 'Field Name'.

tshark -r <trace file> -Y "<whatever combination of filters>" -T fields -e nfs.data | tr -d '\n',':' | xxd -r -ps > out.bin

tshark produces hex dump in a text format with semicolon separators between bytes and newline separators between blocks from individual packets

The " tr -d '\n',':' " removed all newlines and semicolons

the " xxd -r -ps " finally converts continuous stream of hex text dump to binary, which is the same result as "exporting raw" from WireShark.

Rearmost

Masterpiece. Thank you! I was able to sniff GELF this way ;) – Saliferous