Unimplemented type list when trying to write.table

Question

I have the following data.table (data.frame) called output:

> head(output)
        Id                                           Title IsProhibited
1 10000074                             Renault Logan, 2005            0
2 10000124              Ñêëàäñêîå ïîìåùåíèå, 345 ì<U+00B2>            0
3 10000175                                          Ñó-øåô            0
4 10000196             3-ê êâàðòèðà, 64 ì<U+00B2>, 3/5 ýò.            0
5 10000387        Samsung galaxy S4 mini GT-I9190 (÷¸ðíûé)            0
6 10000395 Êàðòèíà ""Êðûì. Ïîñåëîê Àðîìàò"" (õîëñò, ìàñëî)            0

I am trying to export it to a CSV like so:

> write.table(output, 'output.csv', sep = ',', row.names = FALSE, append = T)

However, when doing so I get the following error:

Error in .External2(C_writetable, x, file, nrow(x), p, rnames, sep, eol,  : 
unimplemented type 'list' in 'EncodeElement'
In addition: Warning message:
In write.table(output, "output.csv", sep = ",", row.names = FALSE,  :
  appending column names to file

I have tried converting the Title to a string so that it is no longer of type list like so:

toString(output$Title)

But, I get the same error. My types are:

> class(output)
[1] "data.frame"
> class(output$Id)
[1] "integer"
> class(output$Title)
[1] "list"
> class(output$IsProhibited)
[1] "factor"

Can anyone tell me how I can export my data.frame to CSV?

Another strange thing that I've noticed, is that if I write head(output) my text is not encoded properly (as shown above) whereas if I simply write output$Title[0:3] it will display the text correctly like so:

> output$Title[0:3]
[[1]]
[1] "Renault Logan, 2005"

[[2]]
[1] "Складское помещение, 345 м²"

[[3]]
[1] "Су-шеф"

Any ideas regarding that? Is it relevant to my initial problem?

Edit: Here is my new output:

Id  Title   IsProhibited    
10000074    Renault Logan, 2005 0   
10000124    СкладÑкое помещение, 345 м<U+00B2>    0   
10000175    Су-шеф 0   
10000196    3-к квартира, 64 м<U+00B2>, 3/5 ÑÑ‚.  0   
10000387    Samsung galaxy S4 mini GT-I9190 (чёрный)  0   
10000395    Картина \\"Крым. ПоÑелок Ðромат\"\" (холÑÑ‚     маÑло)"    0
10000594    КальÑн 25 Ñм  0   
10000612    1-к квартира, 45 м<U+00B2>, 6/17 ÑÑ‚. 0   
10000816    Гараж, 18 м<U+00B2>   0   
10000831    Платье    0   
10000930    Карбюраторы К-22И, К-22Г от газ 21 и газ 51 0   

Notice how line ID 10000395 is messed up? It seems to contains quotes of it's own which are messing up the CSV. How can I fix that?

Doggerel

Flatten the list column with paste and try again. – Romney

Looks like unlist(output) may be an option, too. – Doggerel

I don't see unlist as an option here. You can unlist and expand those values to separate new rows, duplicating all of the other columns as required. – Romney

Hmm you're right, unlist isn't working. Can you please provide an example how to flatten the list column with paste? I am not familiar with that approach. I have also tried sapply(output$Title, FUN = paste) – Doggerel

Dumb question...but are you actually putting the result of toString(output$Title) back into output$Title, like output$Title <- toString(output$Title)? – Jerejereld

Answer 1

As mentioned in the comments, you should be able to do something like this (untested) to get "flatten" your list into a character vector:

output$Title <- vapply(output$Title, paste, collapse = ", ", character(1L))

As also mentioned, if you wanted to try the unlist approach, you could "expand" each row by the individual values in output$Title, something like this:

x <- vapply(output$Title, length, 1L)          ## How many items per list element
output <- output[rep(rownames(output), x), ]   ## Expand the data frame
output$Title <- unlist(output$Title, use.names = FALSE)  ## Replace with raw values

Romney

I am not sure how to explain the results - basically, they are unexpected. When I output to CSV, it creates like 100+ columns (I only expect 3 columns). Also, if I do head(output) it outputs a lot of data (almost freezes RStudio). – Doggerel

@user1477388, Oops. I meant vapply not do.call. This is why you should always post a minimal reproducible example! – Romney

@user1477388, vapply is like your sapply approach, which should also work if you add the collapse = ", " part to it. – Romney

Thanks this seems to work well enough. The only strange thing is that it doesn't output my Russian text (the encoding is all messed up, as described in my question above). I am even setting the encoding in code like write.table(output, 'output.csv', sep = ',', row.names = FALSE, append = T, fileEncoding = 'UTF-8') – Doggerel

Do you think I should open a new question regarding the messed up write.csv output? – Doggerel

@user1477388, if you do, please use dput(head(output)) so that others can see exactly what you're working with. – Romney

Answer 2

Do this, irrespective of how many columns you have:

df <- apply(df,2,as.character)

Then do write.csv.

Rust

Why does this help? What is happening and how does this fix it? – Sudanic

This works because it's coercing all of the columns in the data frame to character. The issue is that one of this columns in data frame is stored as a list, which can't be written in a tabular format (or not by default). Setting everything to character "solves" that though it may be introducing unexpected issues and you should really try to figure out why one of the columns is a list in the first place. – Jerejereld

Answer 3

There is a new function (introduced in november 2016) in data.table package that handles writing a data.table object to csv quite well, even in those cases when a column of the data.table is a list.

fwrite(data.table, file ="myDT.csv")

Caudex

this gives me the error: Error in fwrite(DT, file = "myDT.csv") : Row 1 of list column is type 'list' - not yet implemented. fwrite() can write list columns containing atomic vectors of type logical, integer, integer64, double, character and factor, currently. – Abstergent

So you have a list inside a list inside a data.frame. Convert your column to a something easier to convert to a data.frame! – Caudex

@Caudex How would I do that? I'm having trouble with it here. #59997000 – Gandy

Answer 4

Another easy solution. Maybe one or more columns are of type list, so we need convert them to "character" or data frame. So there are two easy solutions

1. Convert each column "as.character" using--

   df$col1 = as.character(df$col1)

   df$col2 = as.character(df$col2)

   .......and so on

2. The best one convert df in to a "matrix"

   df = as.matrix(df)

now write df into csv. Works for me.

Kinin

Answer 5

# First coerce the data.frame to all-character
df = data.frame(lapply(output, as.character), stringsAsFactors=FALSE)

# write file
write.csv(df,"output.csv")

Silures

Answer 6

Those are all elegant solutions.

For the curious reader who would prefer some R-code to ready made packages , here's an R-function that returns a non-list dataframe that can be exported and saved as .csv.

output is the "troublesome" data frame in question.

df_unlist<-function(df){

df<-as.data.frame(df)

nr<-nrow(df)

c.names<-colnames(df)

lscols<-as.vector(which(apply(df,2,is.list)==TRUE))

if(length(lscols)!=0){

for(i in lscols){

temp<-as.vector(unlist(df[,i]))

if(length(temp)!=nr){

adj<-nr-length(temp)

temp<-c(rep(0,adj),temp)

}

df[,i]<-temp

} #end for

df<-as.data.frame(df)

colnames(df)<-c.names
}
return(df)
}

Apply the function on dataframe "output" :

newDF<-df_unlist(output)

You can next confirm that the new (newDF) data frame is not 'listed' via apply(). This should successfully return FALSE.

apply(newDF,2,is.list)         #2 for column-wise step.

Proceed to save the new dataframe, newDF as a .csv file to a path of your choice.

write.csv(newDF,"E:/Data/newDF.csv")

Belomancy

Answer 7

Assuming

• the path you want to save to is Path, i.e. path=Path

• df is the dataframe you want to save,

follow those steps:

1. Save df as txt document:

   write.table(df,"Path/df.txt",sep="|")
   

2. Read the text file into R:

   Data = read.table("Path/df.txt",sep="|")
   

3. Now save as csv:

   write.csv(Data, "Path/df.csv")
   

That's it.

Belomancy

1. still gives me the error: Error in write.table(df, "df.txt", sep = "|") : unimplemented type 'list' in 'EncodeElement' – Abstergent