How do I use tshark to print request-response pairs from a pcap file?

Question

Given a pcap file, I'm able to extract a lot of information from the reconstructed HTTP request and responses using the neat filters provided by Wireshark. I've also been able to split the pcap file into each TCP stream.

Trouble I'm running into now is that of all the cool filters I'm able to use with tshark, I can't find one that will let me print out full request/response bodies. I'm calling something like this:

 tshark -r dump.pcap -R "tcp.stream==123 and http.request" -T fields -e http.request.uri

Is there some filter name I can pass to -e to get the request/response body? The closest I've come is to use the -V flag, but it also prints out a bunch of information I don't necessary want and want to avoid having to kludge out with a "dumb" filter.

Munitions

What was the snarflen of the original capture. If you didnt collect the full packet you probably have the data. – Mesquite

The captures were fine. The MTU on the interface I used was 1514 and I did a capture of 1600. I opened the pcap in Wireshark and can get individual request-response pairs as streams; I was just looking for a way to script against it. – Munitions

Cool - just ruling out the most obvious – Mesquite

What about TShark option -O (-O protocols: Only show packet details of these protocols, comma separated) $ tshark -r clmt_04.pcap -R "http.request or http.response" -V -O http > http.txt – Colossians

Think this would be more useful on SO or SF. – Fox

@Munitions XU: Do you want to use TShark to export the http objects? AFAIK that is not possible at the moment. – Colossians

Answer 1

If you are willing to switch to another tool, tcptrace can do this with the -e option. It also has an HTTP analysis extension (xHTTP option) that generates the HTTP request/repsonse pairs for each TCP stream.

Here is a usage example:

tcptrace --csv -xHTTP -f'port=80' -lten capturefile.pcap

• --csv to format output as comma sperated variable
• -xHTTP for HTTP request/response written to 'http.times' this also switches on -e to dump the TCP stream payloads, so you really don't need -e as well
• -f'port=80' to filter out non-web traffic
• -l for long output form
• -t to give me progress indication
• -n to turn off hostname resolution (much faster without this)

Christy

I used tcptrace. It's pretty promising. Thanks! For some strange reason, just using tcptrace -e my.dump didn't separate out requests correctly. I suspect this is just a case of me doing something wrong since Wireshark does the same splitting just fine, so I'll poke into it a little bit more. If you had a one-liner at the top of your head to extract request-response pairs from a standard pcap file (unfortunately with a handful of cut off packets), I'm all ears :). – Munitions

added an example - this works for me, but I'm sure you will run into issues if yu have truncated packets – Christy

This is perfect for me excepted that I also need the exact timestamp the request got sent. I'm trying to reproduce a scenario where I need to send the stuff at the same time as it happened previously. – Receipt

Answer 2

If you captured a pcap file, you can do the following to show all requests+responses.

filename="capture_file.pcap"
for stream in `tshark -r "$filename" -2 -R "tcp and (http.request or http.response)" -T fields -e tcp.stream | sort -n | uniq`; do
    echo "==========BEGIN REQUEST=========="
    tshark -q -r "$filename" -z follow,tcp,ascii,$stream;
    echo "==========END REQUEST=========="
done;

I just made diyism answer a bit easier to understand (you don't need sudo, and multiline script is imo simple to look at)

Acetophenetidin

Answer 3

This probably wasn't an option when the question was asked but newer versions of tshark can "follow" conversations.

tshark -nr dump.pcap -qz follow,tcp,ascii,123

I know this is a super old question. I'm just adding this for anyone that ends up here looking for a current solution.

Abstracted

Answer 4

I use this line to show last 10 seconds request body and response body(https://gist.github.com/diyism/eaa7297cbf2caff7b851):

sudo tshark -a duration:10 -w /tmp/input.pcap;for stream in `sudo tshark -r /tmp/input.pcap -R "tcp and (http.request or http.response) and !(ip.addr==192.168.0.241)" -T fields -e tcp.stream | sort -n | uniq`; do sudo tshark -q -r /tmp/input.pcap -z follow,tcp,ascii,$stream; done;sudo rm /tmp/input.pcap

Corrianne

Answer 5

There isn't a col named seq or id you can directly use in HTTP responses to connect response and request pairs. However, the server can figure it out by using the TCP layer. The key to connect request and response is the parameter http.response_for.uri, so the answer is below:

tshar -r youfile.cap -T fileds -Y "http.response.code==200" -e ip.src -e ip.dst -e http.response.code -e http.response_for.uri

Grijalva

Answer 6

By the way, GET has no BODY, howver the POST and response has, key parameter is http.file_data, you can use -e to dump it out.

Grijalva