Warning message: line appears to contain embedded nulls [duplicate]

Question

I am trying to read in a list of csv files. These csv files have ";" as its separator. After failing to read in the csv files, I have tried to cut the contents in one of the csv files into several parts and read in the values in each part to see where the problem was caused.

This method worked for me, and I have figured out a working code that works for my data:

y <- data.table(read.table(filenames[i], header = FALSE, sep = ";",
                comment.char = "", fill = TRUE, check.names = FALSE,
                blank.lines.skip = TRUE))

But I have encountered another problem. When I copy and paste the original data in a csv file and run the code it just works fine. However, when I try to run the same code on the original csv file, it gives me the 'embedded nulls' warning.

On the outside, the original data and the copied data look exactly the same, and they are all saved in the csv format. Therefore, it is hard for me to find what is exactly causing the warning and what is the difference between my original csv file and the copied csv file.

The data looks similar to below:

Measurement Reports export file;
;
Comment;Time ;E_MW;E_PF;INV11_ACKW;INV12_ACKW;INV21_ACKW;INV22_ACKW;INV31_ACKW;INV32_ACKW;INV41_ACKW;INV42_ACKW;INV51_ACKW;INV52_ACKW;INV61_ACKW;INV62_ACKW;M1_ATEMP;M1_HUMID;M1_PYRA1S;M1_PYRA2S;M1_PYRA3S;M1_WDIREC;M1_WSPEED;
;
;00:00;-0.02  ;-0.36  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;22.32  ;82.32  ;0.00  ;0.00  ;0.00  ;234.83  ;0.00  ;
;00:01;-0.02  ;-0.36  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;0.00  ;22.26  ;82.57  ;0.00  ;0.00  ;0.00  ;214.93  ;0.00  ;
;
;Sum;-1.41    ;-22.10    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;1330.89    ;5098.24    ;0.00    ;0.00    ;0.00    ;11246.06    ;28.48    ;
;Mean;-0.02    ;-0.37    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;0.00    ;22.18    ;84.97    ;0.00    ;0.00    ;0.00    ;187.43    ;0.47    ;
;

Any help would be appreciated. Thank you.

Appal

It may be an encoding issue such as trying to read a UTF-16 encoded file as if it were not so encoded. Try experimenting with the fileEncoding= argument. – Penholder

If Gabor's suggestion doesn't help, you may need to examine your original file with a "binary editor" to see what characters are actually in there. – Unabridged

Thanks for your help. The fileEncoding= argument was the key to read in my data.:) Both UTF-16LE and UCS-2LE worked for my case, so I decided to use UCS-2LE. – Appal

Answer 1

The solution, as posted in a comment by @G. Grothendieck, was to use the fileEncoding= argument to specify the correct encoding, which turned out to be either UTF16LE or UCS-2LE according to the OP.

Writing this as an answer so that is not lost to the comments.

Jehol

For more information about the specifics encodings supported in R (and the correct syntax to use), go to the Encoding section in the help page ?file – Carrizales

Answer 2

I faced similar issue recently where I received an error message like:

1: In read.table(file = file, header = header, sep = sep, quote = quote, : line 3 appears to contain embedded nulls.

I tried resetting the fileEncoding parameter but failed to remove the warnings. Fortunately, I came across skipNul argument mentioned in this link and setting it to T worked for me.

Floorwalker

In my case, the skipNul option also worked fine. No need to try encodings. – Trula

But in another case (data from a different instrument) it did not. I had to use fileEncoding = "UTF16LE". The lesson is that one needs to know very well the structure of the files... – Trula

Answer 3

I agree with the other users that the fileEncoding argument can help. Be sure to also check that the function and file type are correct. I ran into this error when mistakenly trying to use read.csv() to load in Excel file (the error was resolved quickly when I switched to read.xlsx()).

Werewolf

I so appreciate your reminder to check for a careless mistake like this. – Brost

Answer 4

You just need to create your csv file correctly, do not rename a ".Numbers" file to ".csv". Open your .Numbers file, export it to csv and then type following code in your R environment:

test <- read.csv("MyFile.csv")

Cedric