[R] POSIXlt converted to POSIXct in as.data.frame()

Martin Maechler maechler at stat.math.ethz.ch
Wed Nov 15 16:08:39 CET 2006

Thank you, Spencer.

It *does* depend on the exact environment such as timezone,
locale, ???
In my (Linux) setup, I can reproduce Rogers' 1-off problem
using exactly your (Spencer's) R script.

Carefully considering the warning in  ?POSIXlt,
and using  Sys.getenv("TZ")  to see that I had no timezone set,
I've used 
    Sys.putenv(TZ = "MET")
to correctly set my timezone, but that still did not help,
so I tried
    Sys.putenv(TZ = "UTC+1")
and that *did* solve the problem.
Interestingly, even
    Sys.putenv(TZ = "")
does solve the problem for me.

Here is small function, derived from your codes,
which is useful for testing these settings:
tstD <- function(x.Plt = strptime(paste(9:13, "Nov 2006"), format="%d %b %Y"))
    ## Purpose: Check POSIXlt -> Dataframe (i.e. -> POSIXct) -> Date conversion
    ## ----------------------------------------------------------------------
    ## Arguments: x.Plt:  a correct "POSIXlt" object
    ## ----------------------------------------------------------------------
    ## Author: Martin Maechler, Date: 15 Nov 2006, 10:16

    stopifnot(inherits(x.Plt, "POSIXlt"))
    d0 <- as.Date(x.Plt)
    if(isTRUE(getOption("verbose"))) {
        cat("as.Date(*): "); str(d0) }

    dd <- data.frame(x.Plt)
    ## -> column converted to P..ct -- definitely needed and desired
    dDate <- as.Date(dd[,"x.Plt"])
    stopifnot(dDate - d0 == 0)
    ## else

>>>>> "SpG" == Spencer Graves <spencer.graves at pdf.com>
>>>>>     on Sun, 12 Nov 2006 08:26:09 -0800 writes:

    SpG> I'm not qualified to say much about POSIX, but I haven't seen a 
    SpG> reply to this in almost 3 days, so I'll offer a comment in the hopes 
    SpG> that it might be useful or that someone else might correct any 
    SpG> misstatement I might make: 

    SpG> First, I didn't find a discrepancy. 
    >> sessionInfo()
    SpG> R version 2.4.0 (2006-10-03)
    SpG> i386-pc-mingw32

    SpG> locale:
    SpG> LC_COLLATE=English_United States.1252;LC_CTYPE=English_United 
    SpG> States.1252;LC_MONETARY=English_United 
    SpG> States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252

    SpG> attached base packages:
    SpG> [1] "methods"   "stats"     "graphics"  "grDevices" "utils"     "datasets"
    SpG> [7] "base"    

    SpG> I used the following modification & extension of your example: 

    SpG> my_POSIXlt <- strptime(c("11-09-2006", "11-10-2006", "11-11-2006",
    SpG>                "11-12-2006", "11-13-2006"), "%m-%d-%Y")
    SpG> str(my_POSIXlt)
    SpG> class(my_POSIXlt)
    SpG> my_Date <- as.Date(my_POSIXlt)
    SpG> str(my_Date)
    SpG> myCharacters <- format(my_Date)
    SpG> class(myCharacters)

    SpG> my_DF <- data.frame(my_POSIXlt)
    SpG> str(my_DF)
    SpG> DF_Date <- as.Date(my_DF$my_POSIXlt)
    SpG> str(DF_Date)
    SpG> DF_Date

    SpG> all.equal(DF_Date, my_Date)
    SpG> # TRUE

    SpG> The data.frame function converts POSIXlt, which is a list, to 
    SpG> POSIXct, which is a numeric vector with attributes: 

    >> attributes(my_DF$my_POSIXlt)
    SpG> $class
    SpG> [1] "POSIXt"  "POSIXct"

    SpG> $tzone
    SpG> [1] ""

    >> is.numeric(my_DF$my_POSIXlt)
    SpG> [1] TRUE

    SpG> If you are getting a day shift, I would guess that you might have 
    SpG> a different version of some of the software or a difference in your 
    SpG> locale.  I just did 'update.packages()' yesterday, so if I'm out of date 
    SpG> on something, I hope someone will help me understand and fix the problem. 

    SpG> Beyond this, have you reviewed the ?POSIXt help file plus Gabor 
    SpG> Grothendieck and Thomas Petzoldt. R help desk: Date and time classes in 
    SpG> R. R News, 4(1):29-32, June 2004 (available from www.r-project.org -> 
    SpG> Newsletter)? 

    SpG> Hope this helps. 
    SpG> Spencer Graves

    SpG> Roger Bivand wrote:
    >> In trying to use as.Date(), I've come across the conversion of POSIXlt to 
    >> POSIXct when a POSIXlt variable is included in a data frame:
    >> my_POSIX <- strptime(c("11-09-2006", "11-10-2006", "11-11-2006", 
    >> "11-12-2006", "11-13-2006"), "%m-%d-%Y")
    >> str(my_POSIX)
    >> my_Date <- as.Date(my_POSIX)
    >> str(my_Date)
    >> data <- format(my_Date)
    >> str(data)
    >> my_DF <- data.frame(my_POSIX)
    >> str(my_DF)
    >> DF_Date <- as.Date(my_DF$my_POSIX)
    >> str(DF_Date)
    >> DF_Date
    >> The consequence (for my LC_TIME and machine time zone) is that when 
    >> as.Date() is applied to the data frame column, it dispatches on 
    >> as.Date.POSIXct() not as.Date.POSIXlt(), causing a day shift (actually 60 
    >> minutes, but because as.Date.POSIXct() says floor(), it ends up being a 
    >> whole day). Should data.frame() be changing POSIXlt to POSIXct?

Yes, that was at least a main idea of POSIXct: it can be dealt
with like a numeric vector and hence fits ideally into a data frame.

    >> As as.data.frame.POSIXlt() is written, it says:
    >>> as.data.frame.POSIXlt
    >> function (x, row.names = NULL, optional = FALSE, ...) 
    >> {
    >> value <- as.data.frame.POSIXct(as.POSIXct(x), row.names, 
    >> optional, ...)
    >> if (!optional) 
    >> names(value) <- deparse(substitute(x))[[1]]
    >> value
    >> }
    >> <environment: namespace:base>
    >> which seems a little brutal. Using I() seems to be a work-around:
    >> my_DF <- data.frame(my_POSIXa=I(my_POSIX))
    >> str(my_DF)
    >> class(my_DF$my_POSIXa)
    >> DF_Date <- as.Date(my_DF$my_POSIXa)
    >> str(DF_Date)
    >> DF_Date
    >> In the original problem (conversion of a list read from PostgreSQL to 
    >> a data frame with as.data.frame()), having to know that you need to insert 
    >> I() is perhaps unexpected.
    >> Roger

    SpG> ______________________________________________
    SpG> R-help at stat.math.ethz.ch mailing list
    SpG> https://stat.ethz.ch/mailman/listinfo/r-help
    SpG> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
    SpG> and provide commented, minimal, self-contained, reproducible code.

More information about the R-help mailing list