Unix Command to get the count of lines in a csv file
Asked Answered
T

6

49

Hi I am new to UNIX and I have to get the count of lines from incoming csv files. I have used the following command to get the count.

wc -l filename.csv

Consider files coming with 1 record iam getting some files with * at the start and for those files if i issue the same command iam getting count as 0. Does * mean anything in here and if i get a file with ctrlm(CR) instead of NL how to get the count of lines in that file. gimme a command that solves the issue.

Townscape answered 19/3, 2014 at 16:32 Comment(2)
some files with * at the start does that mean * is the only character in file (OR) the file has other contents?Gaffney
File has other contents but starting character was * (asterisk)Townscape
I
75

The following query helps you to get the count

cat FILE_NAME  | wc -l
Iain answered 8/1, 2015 at 13:53 Comment(1)
This will return an incorrect result for CSV files containing new line characters.Evenhanded
L
38

All of the answers are wrong. CSV files accept line breaks in between quotes which should still be considered part of the same line. If you have either Python or PHP on your machine, you could be doing something like this:

Python

//From stdin
cat *.csv | python -c "import csv; import sys; print(sum(1 for i in csv.reader(sys.stdin)))"

//From file name
python -c "import csv; print(sum(1 for i in csv.reader(open('csv.csv'))))"

PHP

//From stdin
cat *.csv | php -r 'for($i=0; fgetcsv(STDIN); $i++);echo "$i\n";'

//From file name
php -r 'for($i=0, $fp=fopen("csv.csv", "r"); fgetcsv($fp); $i++);echo "$i\n";'

I have also created a script to simulate the output of wc -l: https://github.com/dhulke/CSVCount

Lobito answered 29/11, 2018 at 22:4 Comment(3)
this is the only correct answer for csv files with newlines in certain fields (which is valid)Insubordinate
How is this correct. OP said unix command, not a way to do it in programming language.Microdont
Because Python is present in most unix systems like awk, wc and other programs.Depository
T
13

In case you have multiple .csv files in the same folder, use

cat *.csv | wc -l

to get the total number of lines in all csv files in the current directory. So, -c counts characters and -m counts bytes (identical as long as you use ASCII). You can also use wc to count the number of files, e.g. by: ls -l | wc -l

Torture answered 12/10, 2018 at 20:20 Comment(0)
L
8

wc -l mytextfile

Or to only output the number of lines:

wc -l < mytextfile

Usage: wc [OPTION]... [FILE]...
or:  wc [OPTION]... --files0-from=F
Print newline, word, and byte counts for each FILE, and a total line if
more than one FILE is specified.  With no FILE, or when FILE is -,
read standard input.
  -c, --bytes            print the byte counts
  -m, --chars            print the character counts
  -l, --lines            print the newline counts
  --files0-from=F    read input from the files specified by
                       NUL-terminated names in file F;
                       If F is - then read names from standard input
  -L, --max-line-length  print the length of the longest line
  -w, --words            print the word counts
  --help     display this help and exit
  --version  output version information and exit
Loaf answered 8/11, 2016 at 23:39 Comment(0)
P
1

You can also use xsv for that. It also supports many other subcommands that are useful for csv files.

xsv count file.csv

Pippo answered 22/6, 2022 at 11:18 Comment(0)
F
0

echo $(wc -l file_name.csv|awk '{print $1}')

Fasano answered 7/10, 2022 at 6:36 Comment(1)
Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.Primm

© 2022 - 2024 — McMap. All rights reserved.