Python Scapy wrpcap - How do you append packets to a pcap file?

Question

I have some software that can emulate things like BER and delays on the network. I need a way to test the BER module of the software to make sure it actually works correctly. My solution is to create a program that sends out raw Ethernet frames with the type field set to an unused type. Inside the Ethernet frame is just random bits. For each frame sent out I need to log the frame to a pcap file. On the other side of the network link will be a receiving application that simply writes every packet it sees to its own pcap log. After the test is done running the two pcap logs will be compared to get the BER.

I'm using the python module Scapy and so far its done everything that I need. I can send out raw Ethernet frames with random data and see them in Wireshark. However, I don't know how to get the wrpcap() method to append to the pcap file, instead of overwriting. I know I can write a list of packets to wrpcap, but this application needs to be able to run for an indefinite amount of time and I don't want to have to wait until the application quits to write all of packets sent to the hard drive. As that would be a lot to store in memory, and if something happened I would have to start the test all over from scratch.

My question is: How do I append to a pcap file using scapy instead of overwriting the pcap file? Is it even possible? If not then what module can do what I need?

While looking for something with Scapy's capabilities I ran into dpkt, but I didn't find a lot of documentation for it. Can dpkt do what I'm asking and if so where can I get some good documentation for it?

Dissenter

Answer 1

There is a way to do what you want, but it means either:

• [Memory hog with one big pcap]: Read the existing pcap from disk with rdpcap() into a scapy PacketList() and then writing frames to the PacketList as they are received. You can selectively save intermediate PacketList to the pcap at will, but I don't think there is anything like an append capability in scapy's wrpcap(). As you mentioned, this technique also means that you are keeping the entire PacketList in memory until completion.

• [Glue individual pcap files together]: Only keep small snapshots of packets in memory... you should save pcap snapshots on a per-X-minute basis to disk, and then aggregate those individual files together when the script finishes.

You can combine pcap files in linux with mergecap from the wireshark package... The following command will combine pak1.pcap and pak2.pcap into all_paks.pcap:

mergecap -w all_paks.pcap pak1.pcap pak2.pcap

As for dpkt, I looked through their source, and it might be able to incrementally write packets, but I can't speak for how stable or maintained their code base is... it looks a bit neglected from the commit logs (last commit was January 9th 2011).

Tallith

Answer 2

For posterity, PcapWriter or RawPcapWriter looks to be the easier way to deal with this in scapy 2.2.0. Couldn't find much documentation other than browsing the source though. A brief example:

from scapy.utils import PcapWriter

pktdump = PcapWriter("banana.pcap", append=True, sync=True)

...
pktdump.write(pkt)
...

Deprecatory

such a shame no Pcap writing documentation. The link to sourcecode append parameter: scapy utils @ github. Note that you need to populate pkt variable with pkt=rdpcap(pcap_file), for example. – Breastbeating

Note 2: you want to add a delay for the append packets maybe. If p is every packet in the list pkt, you would want to code p.time = p.time + delay – Breastbeating

Doc is here FTR scapy.readthedocs.io/en/latest/api/… – Spate

Answer 3

The wrpcap() function can be used to append if you include the keyword argument append=True. For example:

pkt = IP()
wrpcap('/path/to/filename.pcap', pkt, append=True)
pkt2 = IP()
wrpcap('/path/to/filename.pcap', pkt2, append=True)

rdpcap('/path/to/filename.pcap')

<filename.pcap: TCP:0 UDP:0 ICMP:0 Other:2>

Side note: wrpcap opens and closes the file handle with each call. If you have an open file handle to the pcap file, it will be closed after a call to wrpcap().

Benzofuran

Answer 4

I think I am following you here as packets are sniffed you would like to have them all written to a single pcap file? While you cannot append to a pcap you can append the packets to list and then write them all at once to the pcap.

I'm not sure if this answers your question or helps at all, if not let me know and I can tweek it to meet your needs. In this example I set the threshold to create a new pcap for ever 500 packets sniffed. Be careful if you run this twice as your pcaps may get over written on the second go.

#!/usr/bin/python -tt

from scapy.all import *

pkts = []
iter = 0
pcapnum = 0

def makecap(x):
    global pkts
    global iter
    global pcapnum
    pkts.append(x)
    iter += 1
    if iter == 500:
        pcapnum += 1
        pname = "pcap%d.pcap" % pcapnum
        wrpcap(pname, pkts)
        pkts = []
        iter = 0

while 1:
    sniff(prn=makecap)

This should give you a little bit of leverage however the last few packets may get lost (lower the value in the if statement to mitigate this.) Suggest using it on both sides at the same time so each pcap should line up, Later on you can use mergepcap as Mike suggests to if you like. Let me know if this works for you.

Mantellone