[R] From POSIXct to numeric and back with time zone

Fri Aug 23 22:22:16 CEST 2013

I disagree with the characterization that setting TZ to anything but UTC yields confusing results. If your time strings do not specify a time offset, then TZ will be assumed, and if the assumed time zone accounts for daylight savings and you don't want it to then you might be disappointed in the results. I work mostly with standard time year round so I often use the Etc/GMT+8 or similar zones when converting time strings. I have not figured out why as.POSIXct accepts a tz argument... only as.POSIXlt seems to pay attention to it. You might want to read https://stat.ethz.ch/pipermail/r-help/2013-August/357939.html for some related discussion.
---------------------------------------------------------------------------
Jeff Newmiller                        The     .....       .....  Go Live...
DCN:<jdnewmil at dcn.davis.ca.us>        Basics: ##.#.       ##.#.  Live Go...
                                      Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/Batteries            O.O#.       #.O#.  with
/Software/Embedded Controllers)               .OO#.       .OO#.  rocks...1k
--------------------------------------------------------------------------- 
Sent from my phone. Please excuse my brevity.

David Winsemius <dwinsemius at comcast.net> wrote:
>
>On Aug 23, 2013, at 3:12 AM, Daniel Haugstvedt wrote:
>
>> I am replying to my own question in case someone else finds this
>tread and needs help with the same problem. Thanks to Mark Leeds for
>helping me on my way. Any errors or flaws are mine since I have
>rewritten most of his comments to make sure I understood them
>correctly.  
>> 
>> First three general recommendations for time zone problems:
>> 
>> 1) When asking time zone related questions always give OS
>information. It does not hurt to give information on version etc.
>either. My system is OSX, lion (10.8.4). Using str(R.Version()) to get
>system information is one option.
>> 
>> str(R.Version())
>> List of 14
>> $ platform      : chr "x86_64-apple-darwin9.8.0"
>> $ arch          : chr "x86_64"
>> $ os            : chr "darwin9.8.0"
>> $ system        : chr "x86_64, darwin 9.8.0"
>> .
>> .
>> .
>> $ version.string: chr "R version 2.15.1 (2012-06-22)"
>> $ nickname      : chr "Roasted Marshmallows
>> 
>> 2)  Before you do ANYTHING with timezones, put Sys.setenv(TZ = "UTC")
>in your .Rprofile ir at the tiop of the code you're working in. 
>Otherwise, if you start trying to convert date time objects to plain
>date objects, things can really get whacked. 
>> 
>> 3) Check that the time zone you are using is valid. 
>> 
>> I am no expert on this, but from what I understand, in OSX a valid
>time zone has the name of one of the files in the folder
>> 
>> /usr/share/zoneinfo,
>> 
>> with some obvious exceptions like the files "iso3166.tab",
>"posixrules" and "zone.tab". It can also be one of the entries in the
>file, /usr/share/zoneinfo/zone.tab.
>> 
>> CET and CEST (daylight savings time) are the time zone my system use
>when nothing is specified. I am sorry for writing ETC in one of the
>lines in the first email.
>> 
>> 
>> Now, to the problem: How do I change from POSIXct to numeric and back
>with another time zone than UTC?
>> 
>> I have tried to simplify the original question and attempted and
>answer. Please correct me if I am wrong.
>> 
>> 
>> 
>> 
>> Sys.setenv(TZ = "UTC")
>> 
>> ## Number of seconds from '1970-01-01 00:00:00 UTC' to '2000-01-30
>00:00:00 CET' not 
>> ## counting leap seconds. Display as CET date
>> tmp = as.POSIXct( '2000-01-30', origin = '1970-01-01' , tz = "CET")
>> 
>> ## Number of seconds from '1970-01-01 00:00:00 UTC' to '2000-01-30
>00:00:00 CET' not 
>> ## counting leap seconds. Display as UTC date
>> tmp2 =as.POSIXct( as.numeric( tmp ),origin = '1970-01-01' , tz =
>"UTC")
>> 
>> ## What I wanted was to go to numeric and back to the original with
>the same time zone. What I got was
>> ## the number of seconds from '1970-01-01 00:00:00 UTC' to
>'2000-01-30 00:00:00 UTC' not 
>> ## counting leap seconds. Display as CET date. Which is 60*60 seconds
>less then I expect.
>> tmp3 = as.POSIXct( as.numeric( tmp ),origin = '1970-01-01' , tz =
>"CET")
>> 
>> ## Solution: Convert to the desired time zone after as.POSIXct has
>been used wit UTC to get the 
>> ## correct number of seconds
>> tmp4 = tmp2
>> attributes(tmp4)$tzone = 'CET'
>> 
>> 
>> tmp
>> [1] "2000-01-30 CET"
>>> tmp2
>> [1] "2000-01-29 23:00:00 UTC"
>>> tmp3
>> [1] "2000-01-29 23:00:00 CET"
>>> tmp4
>> [1] "2000-01-30 CET"
>>> 
>>> as.numeric(tmp)
>> [1] 949186800
>>> as.numeric(tmp2)
>> [1] 949186800
>>> as.numeric(tmp3)
>> [1] 949183200
>>> as.numeric(tmp4)
>> [1] 949186800
>> 
>> 
>> My conclusions are 
>> 1) The tz argument sets the tzone attribute but it also determines
>how the entered date should be interpreted IF the date is entered as a
>string. 
>> 2) If the date is entered as numeric it is assumed to be the number
>of seconds from UTC to UTC and the tz argument is used to add /
>subtract the number of seconds which converts it to the time zone
>specified.
>
>Thank you for this discussion (and I share your pain.) You might want
>to look at the various `as.POSIXct` methods with:
>
>methods(as.POSIXct)
>
>I see 10 methods on my machine at the moment, but some are from the zoo
>package, so you may see a different number. You are describing
>differences in how `as.POSIXct.numeric` behaves versus
>`as.POSIXct.default` which I believe is where a character or factor
>argument ends up after first being passed through `as.POSIXlt`. In
>other parts of your question the behavior of `as.POSIXct.Date`,
>`as.POSIXct.dates`, and `as.POSIXct.default` are illustrated. I have
>had similar difficulties understanding TZ behavior. It would be nice if
>R automagically looked up the current setting of my system timeszone. 
>
>> 
>> Some additional conclusions that I came across while testing a bit.
>The code which made me draw them are attached at the end. 
>> 3) If a time zone is not needed the tz argument does nothing. It sets
>the tzone but it does not change it.
>> 4) The origin is assumed to be UTC regardless of what Sys.timezone()
>say as long as no time zone for the origin is specified. I checked this
>by changing the Sys.timezone() to CET before running the example again.
>
>
>My understanding: All items in a vector need to be in the same TZ. You
>cannot mix the TZ argument within a vector. Hence, it may be better to
>always convert to UTC for storage (and for mental clarity) and only use
>TZ's for the format.POSIXt output.
>
>I am not able to figure out where the canonical translation for TZ
>abbreviations lies on my MacOS 10.8.5 machine. The file at:
>
>?timezone
>Using the example code with:
>tzfile <- "/usr/share/zoneinfo/zone.tab"
>
>... does not have any three letter abbreviations. My value for TZ is
>numbered 390, with a name of  "America/Los_Angeles". My Sys.time
>function behaves properly:
>
>> Sys.time()
>[1] "2013-08-23 09:54:58 PDT"
>
>But: "How do it know?"
>
>If I change my system TZ to US Central Daylight time there is no
>apparent recognition of that fact in the output of Sys.time() ... I
>think I may need to change a locale variable. If I enter a TZ with
>Sys.setenv() I get CDT as the output:
>
>> Sys.setenv(TZ = "America/Chicago")
>> Sys.time()
>[1] "2013-08-23 12:00:11 CDT"
>
>So I can change R's understanding of the local TZ if I use the full
>tzone$name value but offering a value of either PDT or PST will fail
>for the Sys.setenv(TZ=) argument. (Sys.getenv('TZ') returns "" when I
>start R.)
>
>Best of luck in this quest and thanks for the illuminating exercises.
>(And I thought your spelling was as good as most native English
>speakers, certainly better than I generally exhibit, so I think you
>shoudl stop apologizing.)