Use find, wc, and sed to count lines

Question

I was trying to use sed to count all the lines based on a particular extension.

find -name '*.m' -exec wc -l {} \; | sed ...

I was trying to do the following, how would I include sed in this particular line to get the totals.

Dragonhead

Answer 1

You may also get the nice formatting from wc with :

wc `find -name '*.m'`

Savick

Add -type f to avoid the case of a directory name that matches: wc $(find -type f -name '*.m') – Rinaldo

wc $(find -name '*.m') is prettier. – Lynsey

This will break for any file names which contain whitespace or other shell metacharacters. – Casper

@Casper How would you recover from white-spaces in this instance? I am currently have this issue. – Aubade

@Workingdollar mywiki.wooledge.org/BashFAQ/020 and/or several of the other answers on this page. – Casper

@Casper thank you for the documentation. I found that this worked most effectively for me find . -name '*.m' -exec wc {} \; – Aubade

Answer 2

Most of the answers here won't work well for a large number of files. Some will break if the list of file names is too long for a single command line call, others are inefficient because -exec starts a new process for every file. I believe a robust and efficient solution would be:

find . -type f -name "*.m" -print0 | xargs -0 cat | wc -l

Using cat in this way is fine, as its output is piped straight into wc so only a small amount of the files' content is kept in memory at once. If there are too many files for a single invocation of cat, cat will be called multiple times, but all the output will still be piped into a single wc process.

Garpike

Or use the standard/portable form instead: find . -type f -name '*.m' -exec cat {} + | wc -l. – Claar

how to modify this to print out total lines for each file among with the file name? – Kigali

@Kigali Then trivially run -exec wc -l {} + instead of -print0 | xargs .... Using + with -exec might end up running more than one instance of wc -l and then you will need to sum the totals from each run for the overall total. Or if you don't care about totals, just remove those lines with grep -v; or, use -exec wc -l {} \; to run a separate instance of wc on each file, at a somewhat higher processing cost. – Casper

Answer 3

You can cat all files through a single wc instance to get the total number of lines:

find . -name '*.m' -exec cat {} \; | wc -l

Octane

Answer 4

On modern GNU platforms wc and find take -print0 and -files0-from parameters that can be combined into a command that count lines in files with total at the end. Example:

find . -name '*.c' -type f -print0 | wc -l --files0-from=-

Sadducee

Answer 5

you could use sed also for counting lines in place of wc:

 find . -name '*.m' -exec sed -n '$=' {} \;

where '$=' is a "special variable" that keep the count of lines

EDIT

you could also try something like sloccount

Ladylove

find . -name '*.m' -exec sed -n 'where $=' {} \; Is this it? – Dragonhead

That is not the total though, added together. – Dragonhead

OK, I ended up with this. $ find . -name '*.m' -exec sed -n '$=' {} \; | sum - 22696 1 – Dragonhead

where '$=' is a "special variable" that keep the count of lines. You jest sire! $= represents an address and a command. The $ is last line, the command = is current line number, in conjunction with -n switch which supresses pattern space printout. The outcome is it counts the number of lines fed to it. – Warenne

@Ladylove Thanks. How to print the filename before the count? – Kigali

Answer 6

Hm, solution with cat may be problematic if you have many files, especially big ones.

Second solution doesn't give total, just lines per file, as I tested.

I'll prefer something like this:

find . -name '*.m' | xargs wc -l | tail -1

This will do the job fast, no matter how many and how big files you have.

Zeller

If there are too many files for a single command line, xargs will chunk them and this will only give the total for the final chunk. – Garpike

Answer 7

sed is not the proper tool for counting. Use awk instead:

find . -name '*.m' -exec awk '{print NR}' {} +

Using + instead of \; forces find to call awk every N files found (like with xargs).

Pastor

Funny, I actually meant to say awk – Dragonhead

Answer 8

For big directories we should use:

find . -type f -name '*.m' -exec sed -n '$=' '{}' + 2>/dev/null | awk '{ total+=$1 }END{print total}' 

# alternative using awk twice
find . -type f -name '*.m' -exec awk 'END {print NR}' '{}' + 2>/dev/null | awk '{ total+=$1 }END{print total}' 

Criminality