Simplifying a POSIX node with RJSONIO::fromJSON()

Question

I have the following vector of double values, x, where each element represents a POSIX date-time

x <- c(1417621083, 1417621204, 1417621384, 1417621564, 1417621623)

I am using the RJSONIO package, and would like to continue to do so.

As an exercise, I'd like to convert these values into JSON text and then read them back into R again, but am having trouble getting the date-time representations into a nice simplified list result. In JSON, the dates need to be in a special format so the values in x are converted to the following:

dates <- c("/new Date(1417621083)", "/Date(1417621204)", "/Date(1417621384)", 
           "/Date(1417621564)", "/Date(1417621623)")

When I run dates with a second arbitrary vector through the RJSONIO parser, everything seems to go smoothly.

library(RJSONIO)
make <- toJSON(list(date = dates, value = LETTERS))

Then when I parse the new JSON text using the stringFun option with the R-json C routine for dates, the result is a two-element list, the first element being a list and the second an atomic vector.

(read <- fromJSON(make, stringFun = "R_json_dateStringOp"))
# $date
# $date[[1]]
# [1] "2014-12-03 07:38:03 PST"
# 
# $date[[2]]
# [1] "2014-12-03 07:40:04 PST"
# 
# $date[[3]]
# [1] "2014-12-03 07:43:04 PST"
# 
# $date[[4]]
# [1] "2014-12-03 07:46:04 PST"
# 
# $date[[5]]
# [1] "2014-12-03 07:47:03 PST"
# 
# 
# $value
# [1] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M"
# [14] "N" "O" "P" "Q" "R" "S" "T" "U" "V" "W" "X" "Y" "Z"

But I was expecting a list of two vectors, and I would rather have it in the form of

# $date
# [1] "2014-12-03 07:38:03 PST" "2014-12-03 07:40:04 PST"
# [3] "2014-12-03 07:43:04 PST" "2014-12-03 07:46:04 PST"
# [5] "2014-12-03 07:47:03 PST"
# 
# $value
# [1] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M" "N" "O" "P" "Q"
# [18] "R" "S" "T" "U" "V" "W" "X" "Y" "Z"

I tried several ways to simplify the result from within the call to fromJSON(), and none of them have worked. Here are a couple of my attempts:

Using a handler : This simplifies the result but fails to reformat the dates

h1 <- basicJSONHandler(simplify = TRUE)
fromJSON(make, handler = h1, stringFun = "R_json_dateStringOp")
# $date
# [1] "/new Date(1417621083)" "/Date(1417621204)"    
# [3] "/Date(1417621384)"     "/Date(1417621564)"    
# [5] "/Date(1417621623)"    
# 
# $value
# [1] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M"
# [14] "N" "O" "P" "Q" "R" "S" "T" "U" "V" "W" "X" "Y" "Z"

Trying the simplify argument : I tried several different varieties of this and none worked.

fromJSON(make, simplify = StrictCharacter)
# $date
# [1] "/new Date(1417621083)" "/Date(1417621204)"    
# [3] "/Date(1417621384)"     "/Date(1417621564)"    
# [5] "/Date(1417621623)"    
#
# $value
#  [1] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" "K" "L" "M"
# [14] "N" "O" "P" "Q" "R" "S" "T" "U" "V" "W" "X" "Y" "Z"

Is there a way to simplify the result for the dates in the call to fromJSON()?

Melancholia

Answer 1

I think you cannot get the coercion of dates and their simplification to a vector in the same time. For the simple reason that this is not (yet) implemented in RJSONIO. Indeed as you mention the simplification is done using one the flag : StrictLogical, StrictNumeric and StrictCharacter which create logicals, numbers or characters vectors. Maybe you should contact the maintainer to add StrictPosixct flag for POSIXct dates.

Using stringFun can't help because it receives a scalar element(a character string) and it is not aware of other vector elements. You can check this by defining an R function as stringFun parameter and put a browser within it.

convertJSONDate <-
  function(x)
  {
     if(grepl('Date',x)){
       val <- sub('.*[(]([0-9]+).*','\\1',x)
       return(structure(as.numeric(val)/1000, class = c("POSIXct", "POSIXt")))
     }
     x
   }

I guess you want to do the coercion/simplification when you parse your json for a performance reason. I would use a different strategy :

1. I coerce my numeric values to a POSIXct and I will store them as a character in a well formatted dates. This is better then the special ( ugly) "new Date(.. ,date") RJSONIO date format. Remember that json format is a standard format that can be parsed by other languages ( python, js,..)
2. Then parse my dates as a normal character and I use the fast fasttime package to coerce it to POSIXct vector.

here some code to show this:

## coerce x to dates a well formatted dates
dd <- as.character(as.POSIXct(x,origin = '1970-01-01' , tz = "UTC"))
## read it again in a fast way
fastPOSIXct(fromJSON(make)$date)

[1] "2014-12-03 16:38:03 CET" "2014-12-03 16:40:04 CET" "2014-12-03 16:43:04 CET" "2014-12-03 16:46:04 CET" "2014-12-03 16:47:03 CET"

Penrod

Answer 2

From the read value which I'm guessing is the desired starting point... this is one way:

> dd <- sapply(read, c) 
> class(dd) <- "POSIXct"
> dd
[1] "2014-12-03 07:38:03 PST" "2014-12-03 07:40:04 PST" "2014-12-03 07:43:04 PST"
[4] "2014-12-03 07:46:04 PST" "2014-12-03 07:47:03 PST"

Class-coercion is kind of "dirty" but I had already tried quite a few other (failed) strategies, e.g. unlist, sapply( read,"[[",1), sapply(read, c) ), for preserving attributes, so I decided to get down in the mud with R and swing the class-hammer.

Capetian

Answer 3

Since the bounty has expired and it turns out that this cannot currently be done in RJSONIO (or so it seems), I'm going to put this up as another alternative method in case any other users come across this issue and need a solution.

Package jsonlite can do this operation easily. All we have to do is add the POSIXt class to the numeric vector and specify "mongo" for the parser in the non-exported function asJSON.

# unloadNamespace(RJSONIO)  ## to avoid confusion between packages
library(jsonlite)    

x <- c(1417621083, 1417621204, 1417621384, 1417621564, 1417621623)

class(x) <- "POSIXt"    

data <- list(dates = x, values = letters[1:5])

json <- jsonlite:::asJSON(data, POSIXt = "mongo")

fromJSON(json)
# $dates
# [1] "2014-12-03 07:38:03 PST" "2014-12-03 07:40:04 PST"
# [3] "2014-12-03 07:43:04 PST" "2014-12-03 07:46:04 PST"
# [5] "2014-12-03 07:47:03 PST"
# 
# $values
# [1] "a" "b" "c" "d" "e"

Melancholia