blank.lines.skip = TRUE fails with read.fwf?

Question

There are four blank lines at the end of my file.

> data=read.fwf("test2",head=F,widths=c(3,1,-3,4,-1,4),blank.lines.skip = TRUE)  
> data  

When I run this code, the blank.lines.skip argument is ignored. I still get blank lines in my output.

The file is:

x1     F          1890 1962  
x2                1857 1936  
x3                1900 1978  
x4                1902 1994  
x5        F       1878 1939 

and four blank lines at the end.

Gnosticism

Also, please be aware that the manner in which you've asked this particular question is potentially quite frustrating for those of us who'd like to help. We don't have the file you're trying to read, so any help we offer would merely be guesses. That's why we strongly recommend that you make your question reproducible. – Pendent

I appreciate your attempt to clarify your question. However, whatever is going wrong will likely be particular to the way your specific file is stored, which is lost when you simply copy+paste the contents into your question. We will need the file itself. A dropbox link, perhaps? We need to be able to replicate what you're doing exactly. – Pendent

would you mind watching my answer below? – Gnosticism

i know i can delete the blank lines ,but i want to learn something in R. – Gnosticism

Answer 1

It looks like you're right that blank.lines.skip does not apply to read.fwf -- would have to dig in the code to figure out why, but read.fwf does significant processing of the file before passing it (along with the blank.lines.skip directive) to read.table. However, it's not too hard to detect and remove all-blank lines after the fact.

For example:

cat("abc","def","ghi","","","",sep="\n",file="test3.dat")
read.table("test3.dat")  ## blank lines skipped (by default)
(x <- read.fwf("test3.dat",widths=c(1,1,1),blank.lines.skip=TRUE))
##     V1   V2   V3
## 1    a    b    c
## 2    d    e    f
## 3    g    h    i
## 4 <NA> <NA> <NA>
## 5 <NA> <NA> <NA>
## 6 <NA> <NA> <NA>
all_NA <- apply(x,1,function(z) all(is.na(z)))
x[!all_NA,]

To answer your other question (which you posted as an answer, which was then deleted by a moderator; you should generally add necessary modifications to your question by editing your original post, or by commenting on questions if necessary, rather than posting an answer): colClasses is indeed not "clever" enough for you to use automatic detection on most columns but override it for (a) specific column(s).

Pollie

Instead of faking a CSV file by using cat to write out a set of strings, you can directly replace read.fwf( textConnection("abc def ghi\n....\n"), ...) – Protozoon