count (non-blank) lines-of-code in bash

Question

In Bash, how do I count the number of non-blank lines of code in a project?

Crochet

A lot of the solutions below just work for one file (e.g. foo.c). Any thoughts about the toal number of lines in a project (e.g. many files in directory structure, and excluding binary files)? – Broddy

@Broddy I think I can answer that part. For any solution that works on one file, e.g. "cat FILE | sed blah", you can work on many files by replacing the "cat FILE" with a command which lists the filenames to operate on, e.g. "find . -name '*.py'", and pipe that into "xargs cat". e.g. "find . -name '*.py' | xargs cat | sed '/^\s*$/d' | wc -l" – Crochet

@JonathanHartley @Broddy there are also programs like sloc and cloc that are here to do those code lines counts. – Surveyor

OP here: When I first asked this problem, 'cloc' didn't do a very good job on Python code. Nowadays it's great. – Crochet

cloc is also available as an npm module and saves lot of time. – Process

Answer 1

cat foo.c | sed '/^\s*$/d' | wc -l

And if you consider comments blank lines:

cat foo.pl | sed '/^\s*#/d;/^\s*$/d' | wc -l

Although, that's language dependent.

Hippocrates

Not sure why you're using cat there. Use foo.c or foo.pl as the filename to pass to sed. sed '/^\s*$/d' foo.c | wc -l – Undulate

Just habit. I read pipelines from left to right, which means I usually start with cat, then action, action, action, etc. Clearly, the end result is the same. – Hippocrates

To do this for all files in all subfolders and to exclude comments with '//', extend this command into this: find . -type f -name '*.c' -exec cat {} \; | sed '/^\s*#/d;/^\s*$/d;/^\s*\/\//d' | wc -l – Cotillion

@Andy: Give him the Useless Use Of Cat award! – Gash

You can read left to right without UUOC: < foo.pl sed 'stuff' | wc -l. – Ancylostomiasis

Generally speaking, UUOC is not important, but readability is. – Caballero

If there are more than a few files, "find -exec" is slow, as it launches a new instance of cat for each file. It's not even the shell builtin cat, but /bin/cat, so you've got the whole fork/exec going on there for each file. Use "find . -type f | xargs cat | sed", or if you might have spaces in filenames, "find . -type f -print0 | xargs -0 cat | sed". Speaking of which, you really don't even need cat, as sed takes multiple files Thus, "find . -type f -name '.c' | xargs sed -r '/^\s*(\/\/.)?$/d' | wc -l' works just fine (except for the pesky /* */ multiline comments) – Helsie

This worked for me grep -c '^$' $YOURFILE from here. – Sarilda

Please consider adding cat file | wc -l. I noticed the question is looking for non-blank lines; still, for reference it would be nice as this is the first google result for "bash count lines" (at least for me it was). – Eocene

@Eocene Or wc -l file! – Overload

@Sarilda That counts blank lines; to count non-blank, just invert the matching with -v: grep -vc '^$' $YOURFILE. To include whitespace only lines as blank, well, see SpoonMeiser's answer. – Overload

Here is code if you want to cound line for each of the .sh files in current directory separately for f in *.sh; do lines=$(sed '/^\s*#/d;/^\s*$/d' $f | wc -l); echo ${f} : ${lines}; done – Amen

Awesome! For gnu sed do not use \s, use [[:space:]] instead. I.e. sed '/^[[:space:]]*$/d' file | wc -l – Henriques

Answer 2

#!/bin/bash
find . -path './pma' -prune -o -path './blog' -prune -o -path './punbb' -prune -o -path './js/3rdparty' -prune -o -print | egrep '\.php|\.as|\.sql|\.css|\.js' | grep -v '\.svn' | xargs cat | sed '/^\s*$/d' | wc -l

The above will give you the total count of lines of code (blank lines removed) for a project (current folder and all subfolders recursively).

In the above "./blog" "./punbb" "./js/3rdparty" and "./pma" are folders I blacklist as I didn't write the code in them. Also .php, .as, .sql, .css, .js are the extensions of the files being looked at. Any files with a different extension are ignored.

Cerallua

variation for a Rails app: find . -path './log' -prune -o -path './trunk' -prune -o -path './branches' -prune -o -path './vendor' -prune -o -path './tmp' -prune -o -print | egrep '\.rb|\.erb|\.css|\.js|\.yml' | grep -v 'svn' | xargs cat | sed '/^\s*$/d' | wc -l – Cryptanalysis

You need to add a $ to the grep (...\.js$|...) otherwise it will match feature.js.swp. – Whitaker

You forgot the anchoring, so it includes wrong files. And an even simpler version with anchoring: find . | egrep '.\.c$|.\.h$' | xargs cat | sed '/^\s*$/d' | wc -l – Illumine

Answer 3

There are many ways to do this, using common shell utilities.

My solution is:

grep -cve '^\s*$' <file>

This searches for lines in <file> the do not match (-v) lines that match the pattern (-e) '^\s*$', which is the beginning of a line, followed by 0 or more whitespace characters, followed by the end of a line (ie. no content other then whitespace), and display a count of matching lines (-c) instead of the matching lines themselves.

An advantage of this method over methods that involve piping into wc, is that you can specify multiple files and get a separate count for each file:

$ grep -cve '^\s*$' *.hh

config.hh:36
exceptions.hh:48
layer.hh:52
main.hh:39

Leoleod

Thanks! Incidentally, wc does provide a count for each given file, plus a total. – Crochet

Not if you're piping into it though, as standard in counts as just one file. – Leoleod

This is the best answer in my opinion. – Nicolenicolea

-e is not necessary. That's the normal positional location of the pattern and you're not doing anything funky with it. But nothing wrong with being explicit, if that's your style. – Overload

Answer 4

If you want to use something other than a shell script, try CLOC:

cloc counts blank lines, comment lines, and physical lines of source code in many programming languages. It is written entirely in Perl with no dependencies outside the standard distribution of Perl v5.6 and higher (code from some external modules is embedded within cloc) and so is quite portable.

Schroer

When I first asked this question, 'cloc' counted Python docstrings as lines of code, which was suboptimal IMHO. Modern versions of 'cloc' now count Python docstrings as comments, which I like much more. – Crochet

This is the correct answer! I just tried cloc out and it does the job well. – Turning

Answer 5

This command count number of non-blank lines.
cat fileName | grep -v ^$ | wc -l
grep -v ^$ regular expression function is ignore blank lines.

Possum

This answer is the most straightforward – Malisamalison

There is no need for cat in this chain: grep -v ^$ fileName | wl -l – Ihab

There is also no need for wc -l because grep has -c: grep -vc ^$ fileName – Overload

Answer 6

Solution

cat file.txt | awk 'NF' | wc -l

Explanation

'NF' evaluates to true if the number of fields (NF) in a line is non-zero. By default, awk interprets each line of input as records and breaks them into fields based on whitespace. So, 'NF' evaluates to true for lines that are not empty. As a result, this command filters out empty lines from the input.

Flatt

love the simplicity of this one 👏🏼 – Gunzburg

or directly awk 'NF' file.txt | wc -l – Broadloom

Answer 7

'wc' counts lines, words, chars, so to count all lines (including blank ones) use:

wc *.py

To filter out the blank lines, you can use grep:

grep -v '^\s*$' *.py | wc

'-v' tells grep to output all lines except those that match '^' is the start of a line '\s*' is zero or more whitespace characters '$' is the end of a line *.py is my example for all the files you wish to count (all python files in current dir) pipe output to wc. Off you go.

I'm answering my own (genuine) question. Couldn't find an stackoverflow entry that covered this.

Crochet

\W isn't a match for whitespace, it matches non-word characters. It's the opposite of \w, word characters. \W Will match anything that isn't alphanumeric or underscore, and therefore won't do what you claim it does here. You mean \s – Leoleod

Answer 8

cat 'filename' | grep '[^ ]' | wc -l

should do the trick just fine

Zinck

Why use cat and pipe the file into grep, when you can pass the filename as an argument to grep in the first place? – Leoleod

true, it's just an old alias I have around... it does essentially the same as your solution instead of using the inverse – Zinck

Answer 9

grep -cvE '(^\s*[/*])|(^\s*$)' foo

-c = count
-v = exclude
-E = extended regex
'(comment lines) OR (empty lines)'
where
^    = beginning of the line
\s   = whitespace
*    = any number of previous characters or none
[/*] = either / or *
|    = OR
$    = end of the line

I post this becaus other options gave wrong answers for me. This worked with my java source, where comment lines start with / or * (i use * on every line in multi-line comment).

Synthetic

This is a workable solution. Only thing to notice: it doesn't count multi line comments – Funk

Answer 10

awk '/^[[:space:]]*$/ {++x} END {print x}' "$testfile"

Hatband

I'd vote this up just because I've literally never seen anyone use preincrement in an awk script, but unfortunately this only counts the blank lines. :) You mean awk '!/^[[:space:]]*$/{++x} END{print x}'. Or, if you really hate negatives, awk '{y++} /^[[:space:]]*$/{++x} END{print y-x}' ;) – Helsie

Answer 11

Here's a Bash script that counts the lines of code in a project. It traverses a source tree recursively, and it excludes blank lines and single line comments that use "//".

# $excluded is a regex for paths to exclude from line counting
excluded="spec\|node_modules\|README\|lib\|docs\|csv\|XLS\|json\|png"

countLines(){
  # $total is the total lines of code counted
  total=0
  # -mindepth exclues the current directory (".")
  for file in `find . -mindepth 1 -name "*.*" |grep -v "$excluded"`; do
    # First sed: only count lines of code that are not commented with //
    # Second sed: don't count blank lines
    # $numLines is the lines of code
    numLines=`cat $file | sed '/\/\//d' | sed '/^\s*$/d' | wc -l`

    # To exclude only blank lines and count comment lines, uncomment this:
    #numLines=`cat $file | sed '/^\s*$/d' | wc -l`

    total=$(($total + $numLines))
    echo "  " $numLines $file
  done
  echo "  " $total in total
}

echo Source code files:
countLines
echo Unit tests:
cd spec
countLines

Here's what the output looks like for my project:

Source code files:
   2 ./buildDocs.sh
   24 ./countLines.sh
   15 ./css/dashboard.css
   53 ./data/un_population/provenance/preprocess.js
   19 ./index.html
   5 ./server/server.js
   2 ./server/startServer.sh
   24 ./SpecRunner.html
   34 ./src/computeLayout.js
   60 ./src/configDiff.js
   18 ./src/dashboardMirror.js
   37 ./src/dashboardScaffold.js
   14 ./src/data.js
   68 ./src/dummyVis.js
   27 ./src/layout.js
   28 ./src/links.js
   5 ./src/main.js
   52 ./src/processActions.js
   86 ./src/timeline.js
   73 ./src/udc.js
   18 ./src/wire.js
   664 in total
Unit tests:
   230 ./ComputeLayoutSpec.js
   134 ./ConfigDiffSpec.js
   134 ./ProcessActionsSpec.js
   84 ./UDCSpec.js
   149 ./WireSpec.js
   731 in total

Enjoy! --Curran

Parthenopaeus

Answer 12

rgrep . | wc -l

gives the count of non blank lines in the current working directory.

Harlow

The most concise and easy way to count all the non blank lines recursively. Awesome! – Cyclopedia

Answer 13

The neatest command is

grep -vc ^$ fileName

with -c option, you don't even need wc -l

Queenie

Answer 14

It's kinda going to depend on the number of files you have in the project. In theory you could use

grep -c '.' <list of files>

Where you can fill the list of files by using the find utility.

grep -c '.' `find -type f`

Would give you a line count per file.

Amethyst

. matches whitespace. This solution only works if you consider a line containing only whitespace to be non-blank, which it technically is, although it probably isn't what you're after. – Leoleod

Answer 15

Script to recursively count all non-blank lines with a certain file extension in the current directory:

#!/usr/bin/env bash
(
echo 0;
for ext in "$@"; do
    for i in $(find . -name "*$ext"); do
        sed '/^\s*$/d' $i | wc -l ## skip blank lines
        #cat $i | wc -l; ## count all lines
        echo +;
    done
done
echo p q;
) | dc;

Sample usage:

./countlines.sh .py .java .html

Satin

Thanks go to @Andy Lester (+1 on your comment) for the "non-blank" part of the recipe. – Satin

Thanks also to @Michael Cramer (+1 on your post) for originally posting the (slightly more verbose) "non-blank" solution. – Satin

Answer 16

If you want the sum of all non-blank lines for all files of a given file extension throughout a project:

while read line
do grep -cve '^\s*$' "$line"
done <  <(find $1 -name "*.$2" -print) | awk '{s+=$1} END {print s}'

First arg is the project's base directory, second is the file extension. Sample usage:

./scriptname ~/Dropbox/project/src java

It's little more than a collection of previous solutions.

Bard

This one gets the award for the largest number of fork+exec calls by launching grep once per line in each file. ;) – Helsie

Answer 17

grep -v '^\W*$' `find -type f` | grep -c '.' > /path/to/lineCountFile.txt

gives an aggregate count for all files in the current directory and its subdirectories.

HTH!

Mancino

\W is non-word chars; this won't match a line like ${-[*]} + $@, for example. Which is surely valid code somewhere in the world. ;) You mean \s for space. – Helsie

Answer 18

This gives the count of number of lines without counting the blank lines:

grep -v ^$ filename wc -l | sed -e 's/ //g' 

Coben

Answer 19

Try this one:

> grep -cve ^$ -cve '^//' *.java

it's easy to memorize and it also excludes blank lines and commented lines.

Alidis

Lines with whitespace will be counted as code with this solution, as well as lines that are only comments but start with whitespace. – Upolu

Answer 20

Using Perl:

perl -ne '!/^\s*$/ && ++$i; eof && printf "%d\n", $i'
# or
perl -ne '++$i if not /^\s*$/; printf "%d\n", $i if eof'
# or
perl -ne '++$i if ! /^\s*$/; printf "%d\n", $i if eof'
# ...

Where /PATTERN/ is anything one want to skip.

Using bash:

while IFS= read -r n; do 
    ! [[ "$n" =~ ^\s*# ]] && ((++i))
done
printf '%d\n' $i

Dena

Answer 21

There's already a program for this on linux called 'wc'.

Just

wc -l *.c 

and it gives you the total lines and the lines for each file.

Evvy

Hey. 'wc' by itself doesn't search subdirs, and it doesn't filter out blank lines, both explicitly asked for in the question. – Crochet

wc counts blank lines. The OP wants to count non-blank lines. It's true he will want to use wc, but only after it has been stream edited using sed – Marieann