I am looking for some tool that will dump total disk I/O by a single process after it ends. So far my finding is :-
For example, I have some process running in background with PID ####. I need the Total Bytes Written and Read by that process in total after the process ends.Can anybody tell how I can extract this information given a process PID.
Feel free to play with this scribble (myio.sh):
#!/bin/bash
TEMPFILE=$(tempfile) # create temp file for results
trap "rm $TEMPFILE; exit 1" SIGINT # cleanup after Ctrl+C
SECONDS=0 # reset timer
$@ & # execute command in background
IO=/proc/$!/io # io data of command
while [ -e $IO ]; do
cat $IO > "$TEMPFILE" # "copy" data
sed 's/.*/& Bytes/' "$TEMPFILE" | column -t
echo
sleep 1
done
S=$SECONDS # save timer
echo -e "\nPerformace after $S seconds:"
while IFS=" " read string value; do
echo $string $(($value/1024/1024/$S)) MByte/s
done < "$TEMPFILE" | column -t
rm "$TEMPFILE" # remove temp file
Syntax: ./myio.sh <your command>
Examples:
./myio.sh dd if=/dev/zero of=/dev/null bs=1G count=4096./myio.sh dd if=/dev/sda1 of=/dev/null bs=1M count=4096Please change dd's of= in last example only if you know what you are doing.
With this simple script from me you can watch an already running process and its IO.
Syntax: pio.sh PID
#!/bin/bash
[ "$1" == "" ] && echo "Error: Missing PID" && exit 1
IO=/proc/$1/io # io data of PID
[ ! -e "$IO" ] && echo "Error: PID does not exist" && exit 2
I=3 # interval in seconds
SECONDS=0 # reset timer
echo "Watching command $(cat /proc/$1/comm) with PID $1"
IFS=" " read rchar wchar syscr syscw rbytes wbytes cwbytes < <(cut -d " " -f2 $IO | tr "\n" " ")
while [ -e $IO ]; do
IFS=" " read rchart wchart syscrt syscwt rbytest wbytest cwbytest < <(cut -d " " -f2 $IO | tr "\n" " ")
S=$SECONDS
[ $S -eq 0 ] && continue
cat << EOF
rchar: $((($rchart-$rchar)/1024/1024/$S)) MByte/s
wchar: $((($wchart-$wchar)/1024/1024/$S)) MByte/s
syscr: $((($syscrt-$syscr)/1024/1024/$S)) MByte/s
syscw: $((($syscwt-$syscw)/1024/1024/$S)) MByte/s
read_bytes: $((($rbytest-$rbytes)/1024/1024/$S)) MByte/s
write_bytes: $((($wbytest-$wbytest)/1024/1024/$S)) MByte/s
cancelled_write_bytes: $((($cwbytest-$cwbytes)/1024/1024/$S)) MByte/s
EOF
echo
sleep $I
done
This approach works by storing the shell process io counters from /proc/[pid]/io/ in an environment variable before running the process, and then subtracting them after the process ends.
For /proc/[pid]/io counter definitions see:
In this example there are no read_bytes or write_bytes from/to storage:
> export proc_self_io="";while read line; do proc_self_io+="$line "; done < /proc/self/io
> head -c 100M < /dev/urandom > /dev/null
> perl -ane 'BEGIN{%prev=split / /,$ENV{proc_self_io}};printf( qq{%22s %i\n}, $F[0], $F[1]-$prev{$F[0]} ) ' /proc/$$/io
rchar: 104863647
wchar: 104858043
syscr: 12941
syscw: 25608
read_bytes: 0
write_bytes: 0
cancelled_write_bytes: 0
In this example there are write_bytes to storage:
> export proc_self_io="";while read line; do proc_self_io+="$line "; done < /proc/self/io
> head -c 100M < /dev/urandom > /tmp/test.tmp
> perl -ane 'BEGIN{%prev=split / /,$ENV{proc_self_io}};printf( qq{%22s %i\n}, $F[0], $F[1]-$prev{$F[0]} ) ' /proc/$$/io
rchar: 104863684
wchar: 104857948
syscr: 12978
syscw: 25621
read_bytes: 0
write_bytes: 104857600
cancelled_write_bytes: 0
In this example there are no read_bytes or write_bytes from/to storage:
> export proc_self_io="";while read line; do proc_self_io+="$line "; done < /proc/self/io
> cat /tmp/test.tmp > /dev/null
> perl -ane 'BEGIN{%prev=split / /,$ENV{proc_self_io}};printf( qq{%22s %i\n}, $F[0], $F[1]-$prev{$F[0]} ) ' /proc/$$/io
rchar: 104863564
wchar: 104858183
syscr: 859
syscw: 836
read_bytes: 0
write_bytes: 0
cancelled_write_bytes: 0
There are 0 read_bytes because /tmp/test.tmp is all in memory(in core).
> fincore /tmp/test.tmp
RES PAGES SIZE FILE
100M 25600 100M /tmp/test.tmp
dd can be used to perform direct IO from the input file, so now there are read_bytes from storage:
> export proc_self_io="";while read line; do proc_self_io+="$line "; done < /proc/self/io
> dd if=/tmp/test.tmp of=/dev/null iflag=direct
204800+0 records in
204800+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 5.77554 s, 18.2 MB/s
> perl -ane 'BEGIN{%prev=split / /,$ENV{proc_self_io}};printf( qq{%22s %i\n}, $F[0], $F[1]-$prev{$F[0]} ) ' /proc/$$/io
rchar: 104863573
wchar: 104858200
syscr: 204868
syscw: 204852
read_bytes: 104857600
write_bytes: 0
cancelled_write_bytes: 0
dd can be used to perform direct IO from the input file and to the output file:
> export proc_self_io="";while read line; do proc_self_io+="$line "; done < /proc/self/io
> dd if=/tmp/test.tmp of=/tmp/test2.tmp iflag=direct oflag=direct
204800+0 records in
204800+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 11.9083 s, 8.8 MB/s
> perl -ane 'BEGIN{%prev=split / /,$ENV{proc_self_io}};printf( qq{%22s %i\n}, $F[0], $F[1]-$prev{$F[0]} ) ' /proc/$$/io
rchar: 104863754
wchar: 104858310
syscr: 205049
syscw: 204863
read_bytes: 104857600
write_bytes: 104857600
cancelled_write_bytes: 0
> rm /tmp/test.tmp /tmp/test2.tmp
© 2022 - 2024 — McMap. All rights reserved.
/proc/<PID>/ioand https://mcmap.net/q/396912/-what-do-the-counters-in-proc-pid-io-mean – Hays