Export pcap data to csv: timestamp, bytes, uplink/downlink, extra info [closed]

Question

I was wondering if there is any tool that can parse pcap data and convert it to a csv file with the following information:

timestamp, bytes, uplink/downlink, some extra info..

Basically, the uplink/downlink could be seen by the IP/MAC address, and the extra info is not really needed, but what I mean with that is choose a specific field of a packet for example.

I have been trying some tools but I have not found the suitable one yet. Otherwise I will write a small parser. Thanks in advance!

Engineman

tshark must do it, I will post if I find the result. – Engineman

Here is an answer on how to use tshark to write .csv files: #6962633 – Impress

Answer 1

TShark
Here are some examples:

$ tshark -r test.pcap -T fields -e frame.number -e eth.src -e eth.dst -e ip.src -e ip.dst -e frame.len > test1.csv

$ tshark -r test.pcap -T fields -e frame.number -e eth.src -e eth.dst -e ip.src -e ip.dst -e frame.len -E header=y -E separator=, > test2.csv

$ tshark -r test.pcap -R "frame.number>40" -T fields -e frame.number -e frame.time -e frame.time_delta -e frame.time_delta_displayed -e frame.time_relative -E header=y > test3.csv

$ tshark -r test.pcap -R "wlan.fc.type_subtype == 0x08" -T fields -e frame.number -e wlan.sa -e wlan.bssid > test4.csv

$ tshark -r test.pcap -R "ip.addr==192.168.1.6 && tcp.port==1696 && ip.addr==67.212.143.22 && tcp.port==80" -T fields -e frame.number -e tcp.analysis.ack_rtt -E header=y > test5.csv

$ tshark -r test.pcap -T fields -e frame.number -e tcp.analysis.ack_rtt -E header=y > test6.csv

Bewray

Answer 2

Look no further, wireshark is your best friend. It can open your pcap file and allow you to specify extra columns which you want. After this you can simply export them as csv. On the main interface, simply right on any one of the columns and select "column preference". This opens a new window which is very intuitive. Just add a new column and specify the field name. As simple as that.

I had tried tshark but trust me it becomes a bit annoying especially with this:

 tshark: Read filters were specified both with "-R" and with additional command-line arguments."

This message pops up if you include too many columns or for whatever unknown reason.

Aaronson

Answer 3

It looks like you want Bro's connection logs:

bro -r trace.pcap
head conn.log

Output:

#separator \x09
#set_separator  ,
#empty_field    (empty)
#unset_field    -
#path   conn
#fields ts  uid id.orig_h   id.orig_p   id.resp_h   id.resp_p   proto   service duration    orig_bytes  resp_bytes  conn_state  local_orig  missed_bytes    history orig_pkts   orig_ip_bytes   resp_pkts   resp_ip_bytes
#types  time    string  addr    port    addr    port    enum    string  intervacount    count   string  bool    count   string  count   count   count   count
1258531221.486539   gvuu4KIHDph 192.168.1.102   68  192.168.1.1 67  udp -   0.163820    301 300 SF  -   0   Dd  1   329 1   328
1258531680.237254   6nWmFGj6kWg 192.168.1.103   137 192.168.1.255   137 udp dns 3.780125    350 0   S0  -   0   546 0   0
1258531693.816224   y2lMKyrnnO6 192.168.1.102   137 192.168.1.255   137 udp dns 3.748647    350 0   S0  -   0   546 0   0

Now parse the relevant fields:

bro-cut ts id.orig_h id.orig_p id.resp_h id.resp_p service orig_bytes resp_bytes < conn.log | head

1258531221.486539   192.168.1.102   68  192.168.1.1     67  -   301 300
1258531680.237254   192.168.1.103   137 192.168.1.255   137 dns 350 0
1258531693.816224   192.168.1.102   137 192.168.1.255   137 dns 350 0
1258531635.800933   192.168.1.103   138 192.168.1.255   138 -   560 0
1258531693.825212   192.168.1.102   138 192.168.1.255   138 -   348 0
1258531803.872834   192.168.1.104   137 192.168.1.255   137 dns 350 0
1258531747.077012   192.168.1.104   138 192.168.1.255   138 -   549 0
1258531924.321413   192.168.1.103   68  192.168.1.1     67  -   303 300
1258531939.613071   192.168.1.102   138 192.168.1.255   138 -   -   -
1258532046.693816   192.168.1.104   68  192.168.1.1 67  -   311 300

Kingfisher

Looks like bro is now zeek - home page – Almaalmaata

Answer 4

Install argus via terminal

sudo apt-get install argus-client

Convert .pcap to .argus file format

argus -r filename.pcap -w filename.argus  

-r <FILE> Read FILE  
-w <FILE> Write FILE  

Convert .argus to .csv file forrmat while choosing which features to extract

ra -r filename.argus -u -s <features-comma-seprated>

Example:
ra -r filename.argus -u -s rank, stime, ltime, dur

-r <FILE> Read FILE
-u Print time values using Unix time format (seconds from the Epoch).
-s Specify the fields to print.

The list of available fields to print can be found here

This information is copied from my original blog which you can read here

Cowl

Answer 5

You can do this from the Wireshark application itself:

• Make sure you have saved the file to disk already (File>Save) (if you have just done a capture)
• Go to File>Export Packet Dissesctions>as "CSV" [etc]
• Then enter a filename (make sure you add .csv on the end as WS does not do this!)

Voila

Bendy

Answer 6

Here is the python tool to divide the pcap into flows and output the extracted features into a CSV file

Try using flows_to_weka tool in python

This requires a version of scapy installed in your system and better to copy the scapy folder inside the weka folder. And copy the wfe.py, tcp_stream.py and entropy.py files inside the scapy folder. After you done this Your current directory should look something like this:

C:\Users\INKAKA\flows_to_weka\scapy

and copy the .pcap file into this folder and try running this command :

$python  wfe.py -i input.pcap -t csv > output.csv

and you can also retrieve the features that you want by adding the required features in tcp_stream.py and wfe.py.

For reference you can visit : https://github.com/fichtner/flows_to_weka

Epos

Answer 7

As noted in the comments to the question, to output the ip addresses for frames in a capture file in csv format use something like:

tshark -r <filename> -t fields -e ip.addr

See the tshark help for more information about options to set the separator and quoting characters in the csv output.

Field names can be determined by using Wireshark to examine the capture file and selecting a particular field in the details pane. The field name will be then shown in the status line at the bottom of the Wireshark window.

Sort

Answer 8

Is it possible that we can set fields separator other than comma ? Because in my PCap file, if i set the separator=, then my data in output file (.csv) doesn't looks good because i have , in my most of the columns.

So i want to know that is there any way we can set the field separator like other charactors i.e., | (pip) etc

Thanks

Resolutive