Modifying timezone of a POSIXct object without changing the display

Question

I have a POSIXct object and would like to change it's tz attribute WITHOUT R to interpret it (interpret it would mean to change how the datetime is displayed on the screen).

Some background: I am using the fasttime package from S.Urbanek, which take strings and cast it to POSIXct very quickly. Problem is that the string should represent a datetime in "GMT" and it's not the case of my data.

I end up with a POSIXct object with tz=GMT, in reality it is tz=GMT+1, if I change the timezone with

attr(datetime, "tzone") <- "Europe/Paris";
datetime  <- .POSIXct(datetime,tz="Europe/Paris"); 

then it will be "displayed" as GMT+2 (the underlying value never change).

EDIT: Here is an example

datetime=as.POSIXct("2011-01-01 12:32:23.234",tz="GMT")
attributes(datetime)
#$tzone
#[1] "GMT"
datetime
#[1] "2011-01-01 12:32:23.233 GMT"

How can I change this attribute without R to interpret it aka how can I change tzone and still have datetime displayed as "2011-01-01 12:32:23.233" ?

EDIT/SOLUTION, @GSee's solution is reasonably fast, lubridate::force_tz very slow

datetime=rep(as.POSIXct("2011-01-01 12:32:23.234",tz="GMT"),1e5)
f <- function(x,tz) return(as.POSIXct(as.numeric(x), origin="1970-01-01", tz=tz))
> system.time(datetime2 <- f(datetime,"Europe/Paris"))
   user  system elapsed 
   0.01    0.00    0.02 
> system.time(datetime3 <- force_tz(datetime,"Europe/Paris"))
   user  system elapsed 
   5.94    0.02    5.98 
identical(datetime2,datetime3)
[1] TRUE

Thigpen

Possibly useful blog post: blog.revolutionanalytics.com/2009/06/converting-time-zones.html – He

lubridate::force_tz – Hephzipa

Newer users should note that you can not have more than one time zone in a single column in a data.frame. – Lian

Note that your function f doesn't correspond to @GSee's latest answer, because the origins won't necessarily be the same. When I do f(datetime[1], tz="Europe/Paris") with your f I get 2011-01-01 13:32:23 CET. So in f I think you should have origin = as.POSIXct("1970-01-01", tz=tz). Also, force_tz appears to be much faster now. – Omen

Answer 1

EDITED:

My previous solution was passing a character value to origin (i.e.origin="1970-01-01"). That only worked here because of a bug (#PR14973) that has now been fixed in R-devel.

origin was being coerced to POSIXct using the tz argument of the as.POSIXct call, and not "GMT" as it was documented to do. The behavior has been changed to match the documentation which, in this case, means that you have to specify your timezone for both the origin and the as.POSIXct call.

datetime
#[1] "2011-01-01 12:32:23.233 GMT"
as.POSIXct(as.numeric(datetime), origin=as.POSIXct("1970-01-01", tz="Europe/Paris"),
           tz="Europe/Paris")
#[1] "2011-01-01 12:32:23.233 CET"

This will also works in older versions of R.

He

Will that do what the OP wants? What about differences in daylight saving between timezones? I can see that a datetime in DST in one time zone would get displayed an hour off when converted a different time zone that wasn't in DST at that point. – Quito

DST shouldn't matter because the original time is GMT (which doesn't have DST), and the goal is to keep the time the same and just pretend like it's a different timezone, IIUC – He

I think that would do it... but this construct a new object, isn't it possible to access the tz member without letting R change the way the datetime is displayed ? – Thigpen

@statquant, you'd think so; I'm having trouble finding a way to do it though. – He

You know what it's ok, It is fast enough, I do not know how R is coded internally but you can think that when the tz attribute is changed an event is raised so the displayed is changed...so may be it is just not possible – Thigpen

This solution does work for the pair "GMT", "Europe/Paris" but not for "EST", "UTC" (R-2.15.3, 64bit, Win7). However, force_tz gives the correct result. – Roulette

@Roulette You need to read ?timezone. "Beware that some of these designations may not be what you think: in particular ‘EST’ is a time zone used in Canada without daylight savings time, and not ‘EST5EDT’ nor (Australian) Eastern Standard Time." – He

You can also replace the pair ("EST","UTC") by ("America/New_York","UTC") - same problem, i.e., for these pairs the displayed time gets changed. BTW: "America/New_York" is automatically translated to "EST" (eastern standard time) if the date is in winter or "EDT" if it is in summer. Indeed, "EST" is without daylight savings time (as expected, see wikipedia). But that is not the problem here. Just try your code with the timezone pairs I have provided. As I have pointed out, force_tz gives the correct result. – Roulette

@Roulette I see. You are correct. Here's an alternate way which is part of what lubridate is doing, I think: x <- as.POSIXlt(datetime); attr(x, "tzone") <- "UTC"; as.POSIXct(x) – He

Cool! Works as far as I can say. The whole timezone conversion issue seems to be a bit of a mess in R. However, I do not know whether this is a problem with R or me not digging deep enough into the matter. But I guess it's the latter. :) – Roulette

Answer 2

To change the tz attribute of a POSIXct variable it is not best practice to convert to character or numeric and then back to POSIXct. Instead you could use the force_tz function of the lubridate package

library(lubridate)

datetime2 <- force_tz(datetime, tzone = "CET")
datetime2
attributes(datetime2)

Makell

nope, I meant to change the tz attribute without changing the way the datetime is displayed – Thigpen

Ah ok, I changed the answer. It's quite easy with the lubridate package. – Makell

damn it's very slow, see my EDIT ! – Thigpen

lubridate:::force_tz is slow because it's making at least 7 calls to as.POSIXlt! I'm guessing you say that it is not best practice to convert to numeric and back because of what looks like a bug. If that is not a bug then you may have a point; otherwise, POSIXct objects are already numeric, so converting to POSIXlt (a list) several times is certainly not best practice. – He

You are right, but besides that I think it is less error-prone. By that I mean you can easily make a mistake when specifying the origin. – Makell

After looking closer, force_tz converts the input to as.POSIXlt, then makes the subsequent as.POSIXlt calls on the POSIXlt object -- so it's not converting from numeric to list several times which is good. There's still lots of method dispatch overhead, but it's definitely better than I thought at first glance. – He

Answer 3

An alternative to the lubridate package is via conversion to and back from character type:

recastTimezone.POSIXct <- function(x, tz) return(
  as.POSIXct(as.character(x), origin = as.POSIXct("1970-01-01"), tz = tz))

(Adapted from GSee's answer)

Don't know if this is efficient, but it would work for time zones with daylight savings.

Test code:

x <- as.POSIXct('2003-01-03 14:00:00', tz = 'Etc/UTC')
x
recastTimezone.POSIXct(x, tz = 'Australia/Melbourne')

Output:

[1] "2003-01-03 14:00:00 UTC"
[1] "2003-01-03 14:00:00 AEDT" # Nothing is changed apart from the time zone.

Output if I replaced as.character() by as.numeric() (as GSee had done):

[1] "2003-01-03 14:00:00 UTC"
[1] "2003-01-03 15:00:00 AEDT" # An hour is added.

Battled

This was the best answer for me on this page. I don't originally have my times in POSIXct format, so I can use something like as.POSIXct(datetime.character.format, format = "%Y%m%d %H:%M", tz = "EST5EDT", origin = "1970-01-01") – Lian