Save a tibble with list columns to disk

Question

I would like to save to disk a tibble that has list-columns (for later use inside R only). Ideally I'd like a fast binary format like feather, however, it doesn't seem to support list cols:

test <- tibble(a= list(c(1,2), c(3,4)))
feather::write_feather(test, 'test.csv')

Error in writeFeather(x, path) : Not implemented: a is a list

I was expecting the methods in the readr package to be able to handle this, but none of the ones I've tried seem to be able to.

How do I do this?

Tensile

Answer 1

You can use saveRDS and readRDS functions:

library(tibble)
test <- tibble(a= list(c(1,2), c(3,4)))
saveRDS(test, "c:/test.rds")
test_2 <- readRDS("c:/test.rds"))
identical(test, test_2)

In the readr package there are read_rds and write_rds functions, which even allow compression to be set.

Truong

Sadly when you read a tibble saved in a RDS file, strings columns are read like factor data type – Moonwort

Answer 2

My experience of the tidyverse is that they do not work with columns containing lists. For example, filter from dplyr does not work correctly for lists inside data.frames. So, for the operations that are not supported you are stuck with functions that do support this.

If you are simply looking for a way to store any R object on disk, I would recommend you check out save or saveRDS (and load and readRDS). This serializes R objects to a binary format. Note that this is only useful as storage between R sessions, and is not interoperable with other analysis tools such as Python or SPSS.

Dunnage

Are you referring to base R's write.csv? Because that also gave me an error: unimplemented type 'list' in 'EncodeElement'. I guess for now I'll have to normalize my data. – Tensile

I was referencing that, but I agree even that is not a good format. I think a simple binary format would be to use saveRDS. This is however not interchangeable with other software packages, and is purely for use between R sessions. – Dunnage

And indeed, you can also reduce your values to a single one. – Dunnage

You can use filter on list.frames, you just have to use the mutate/map step to apply the filter. – Leicestershire

@Paul Hiemstra: Thanks, reducing to a single value wouldn't have worked well in this case, but saveRDS was exactly what I was looking for. Quite surprised I've never encountered it before. :) – Tensile