total size of group of files selected with 'find'

Question

For instance, I have a large filesystem that is filling up faster than I expected. So I look for what's being added:

find /rapidly_shrinking_drive/ -type f -mtime -1 -ls | less

And I find, well, lots of stuff. Thousands of files of six-seven types. I can single out a type and count them:

find /rapidly_shrinking_drive/ -name "*offender1*" -mtime -1 -ls | wc -l

but what I'd really like is to be able to get the total size on disk of these files:

find /rapidly_shrinking_drive/ -name "*offender1*" -mtime -1 | howmuchspace

I'm open to a Perl one-liner for this, if someone's got one, but I'm not going to use any solution that involves a multi-line script, or File::Find.

Rockabilly

Answer 1

Since OP specifically said:

I'm open to a Perl one-liner for this, if someone's got one, but I'm not going to use any solution that involves a multi-line script, or File::Find.

...and there's none yet, here is the perl one-liner:

find . -name "*offender1*" | perl -lne '$total += -s $_; END { print $total }'

Macguiness

Answer 2

The command du tells you about disk usage. Example usage for your specific case:

find rapidly_shrinking_drive/ -name "offender1" -mtime -1 -print0 | du --files0-from=- -hc | tail -n1

(Previously I wrote du -hs, but on my machine that appears to disregard find's input and instead summarises the size of the cwd.)

Pudendum

Very nice. Although, remembering that brutal keyword to 'find' ("--files0-from=") may not actually be easier than remembering the 'awk' sequence. – Rockabilly

Using du version 8.13 the following gives me the same result: du -ch /rapidly_shrinking_drive/*offender1* | tail -n1 – Darvon

My machine doesn't like the --files0-from= option. ;-/ – Goodell

An alternative that seems to work: find rapidly_shrinking_drive/ -name "offender1" -mtime -1 -print0 | xargs -0 du -hc | tail -n1 – Goodell

Note: The flag --files0-from argument in the above is not a spelling mistake. – Sturrock

Answer 3

Darn, Stephan202 is right. I didn't think about du -s (summarize), so instead I used awk:

find rapidly_shrinking_drive/ -name "offender1" -mtime -1 | du | awk '{total+=$1} END{print total}'

I like the other answer better though, and it's almost certainly more efficient.

Timotheus

Alternative using -exec in the find : find rapidly_shrinking_drive/ -name "offender1" -mtime -1 -exec du {} \; | awk '{ total += $1 }END{ print total }' – Ut

is it able to convert the final output number into more human readable format like 103M? – Barger

@zen you can use numfmt (parts of coreutils) and do something like find rapidly_shrinking_drive/ -name "offender1" -mtime -1 | du | awk '{total+=$1} END{print total}' | numfmt --to=si – Cressy

Answer 4

with GNU find,

 find /path -name "offender" -printf "%s\n" | awk '{t+=$1}END{print t}'

Percolator

+1 for the explicit mention of GNU find. (To bad it is less portable that way). – Wickliffe

Answer 5

I'd like to promote jason's comment above to the status of answer, because I believe it's the most mnemonic (though not the most generic, if you really gotta have the file list specified by find):

$ du -hs *.nc
6.1M  foo.nc
280K  foo_region_N2O.nc
8.0K  foo_region_PS.nc
844K  foo_region_xyz.nc
844K  foo_region_z.nc
37M   ETOPO1_Ice_g_gmt4.grd_region_zS.nc
$ du -ch *.nc | tail -n 1
45M total
$ du -cb *.nc | tail -n 1
47033368  total

Ferreous

Answer 6

Recently i faced the same(almost) problem and i came up with this solution.

find $path -type f -printf '%s '

It'll show files sizes in bytes, from man find:

-printf format
    True; print format on the standard output, interpreting `\' escapes and `%' directives.  Field widths and precisions can be spec‐
    ified as with the `printf' C function.  Please note that many of the fields are printed as %s rather than %d, and this  may  mean
    that  flags  don't  work as you might expect.  This also means that the `-' flag does work (it forces fields to be left-aligned).
    Unlike -print, -printf does not add a newline at the end of the string.
    ...
    %s  File's size in bytes.
    ...

And to get a total i used this:

echo $[ $(find $path -type f -printf %s+)0] #b
echo $[($(find $path -type f -printf %s+)0)/1024] #Kb
echo $[($(find $path -type f -printf %s+)0)/1024/1024] #Mb
echo $[($(find $path -type f -printf %s+)0)/1024/1024/1024] #Gb

Digamma

Interesting and uses only find functionality! Nice! Can you please explain the $[..] syntax for echo ? – Fran

@CiprianTomoiagă sure it's bash math syntax echo $[1+1] will print 2 – Digamma

Answer 7

I have tried all this commands but no luck. So I have found this one that gives me an answer:

find . -type f -mtime -30 -exec ls -l {} \; | awk '{ s+=$5 } END { print s }'

Helladic

Answer 8

You could also use ls -l to find their size, then awk to extract the size:

find /rapidly_shrinking_drive/ -name "offender1" -mtime -1 | ls -l | awk '{print $5}' | sum

Sheaff

If you're going to do it this way, you need to use xargs – Protection