[R] subtracting 100 from strptime year vector generates missing values in POSIXct where none appear to exist in strptime year vector
Jonathan Williams
jonathan.williams at dpag.ox.ac.uk
Wed Feb 24 00:52:27 CET 2010
Thanks Don MacQueen for this reply to my initial query - please SEE MY REPLIES TO THESE IDEAS AND FURTHER INFORMATION BELOW
>From: Don MacQueen [macq at llnl.gov]
>Sent: 23 February 2010 21:25
>To: Jonathan Williams; r-help at r-project.org
>
>Subject: Re: [R] Problem with strptime generating missing values where none appear to exist
>
>What happens if you do all that NA checking on dob *before* subtracting 100 from dob$year?
>
>What happens if you use difftime() before subtracting the 100?
>
>Do you get any NAs if you convert dob to POSIXct?
>
>(these are just investigative ideas, obviously)
>
>-Don
==============================================================================
>What happens if you use difftime() before subtracting the 100?
Good thought - if I use difftime before subtracting 100 from dob$year, then there are no missing values!
But, it is not at all obvious to me why this should be so. Here are dob$years for the dates that go through OK after subtracting 100:-
> table(dob$year[!is.na(difftime(sdate,dob))])
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 42 43 46 48
2 12 18 20 24 32 40 52 44 16 30 20 40 62 41 46 60 33 15 16 28 21 23 16 16 4 4 4 4
and now here are the values for the dates that generate missing values:-
> table(dob$year[is.na(difftime(sdate,dob))])
17 23 25 27 28 29 30 31 38 39 40
7 4 13 8 6 9 4 4 4 7 3
I see no obvious differences that could account for the generation of missing values - all of the years that generate missing values are represented in those that don't!
Converting dob and sdate to POSIXct does not make any difference to the basic problem:-
dob[is.na(difftime(as.POSIXct(sdate),as.POSIXct(dob)))]
[1] "1927-04-03" "1927-04-03" "1927-04-03" "1927-04-03" "1925-04-11" "1925-04-11" "1925-04-11" "1925-04-11" "1925-04-11"
[10] "1939-04-03" "1939-04-03" "1939-04-03" "1940-12-30" "1940-12-30" "1940-12-30" "1917-10-14" "1917-10-14" "1917-10-14"
[19] "1917-10-14" "1925-04-16" "1925-04-16" "1925-04-16" "1925-04-16" "1927-04-05" "1927-04-05" "1927-04-05" "1927-04-05"
[28] "1939-04-08" "1939-04-08" "1939-04-08" "1939-04-08" "1938-10-24" "1938-10-24" "1938-10-24" "1938-10-24" "1930-10-16"
[37] "1930-10-16" "1930-10-16" "1930-10-16" "1923-04-17" "1923-04-17" "1923-04-17" "1923-04-17" "1929-04-17" "1929-04-17"
[46] "1929-04-17" "1929-04-17" "1929-04-17" "1925-04-11" "1925-04-11" "1925-04-11" "1925-04-11" "1931-04-02" "1931-04-02"
[55] "1931-04-02" "1931-04-02" "1929-04-18" "1929-04-18" "1929-04-18" "1929-04-18" "1917-10-22" "1917-10-22" "1917-10-22"
[64] "1928-03-28" "1928-03-28" "1928-03-28" "1928-04-09" "1928-04-09" "1928-04-09"
One good thing, though - the missing values (however they arise) are at least apparent in as.POSIXct(dob), where they are invisible in strptime(as.character(BDT),'%d-%b-%y'):-
> strptime(as.character(BDT),'%d-%b-%y')
[1] "2022-07-14" "2022-07-14" "2022-07-14" "2022-07-14" "2021-03-23" "2021-03-23" "2021-03-23" "2027-08-27" "2027-08-27"
[10] "2027-08-27" "2027-08-27" "2040-04-05" "2040-04-05" "2040-04-05" "2040-04-05" "2023-12-15" "2023-12-15" "2023-12-15"
[19] "2023-12-15" "2017-08-19" "2017-08-19" "2017-08-19" "2017-08-19" "2017-08-31" "2017-08-31" "2017-08-31" "2017-08-31"
[28] "2031-05-12" "2031-05-12" "2031-05-12" "2031-05-12" "2031-05-07" "2031-05-07" "2031-05-07" "2031-05-07" "2026-12-31"
[37] "2026-12-31" "2026-12-31" "2026-12-31" "2037-08-20" "2037-08-20" "2037-08-20" "2037-08-20" "2033-12-08" "2033-12-08"
[46] "2033-12-08" "2033-12-08" "2038-07-17" "2038-07-17" "2038-07-17" "2038-07-17" "2020-10-09" "2020-10-09" "2020-10-09"
[55] "2020-10-09" "2025-04-29" "2025-04-29" "2025-04-29" "2025-04-29" "2024-07-03" "2024-07-03" "2024-07-03" "2024-07-03"
[64] "2030-09-21" "2030-09-21" "2030-09-21" "2030-09-21" "2023-08-03" "2023-08-03" "2023-08-03" "2023-08-03" "2024-05-10"
[73] "2024-05-10" "2024-05-10" "2024-05-10" "2038-05-31" "2038-05-31" "2038-05-31" "2038-05-31" "2028-08-23" "2028-08-23"
[82] "2028-08-23" "2028-08-23" "2031-11-19" "2031-11-19" "2022-12-12" "2022-12-12" "2022-12-12" "2022-12-12" "2023-09-14"
[91] "2023-09-14" "2023-09-14" "2023-09-14" "2021-01-12" "2021-01-12" "2021-01-12" "2021-01-12" "2021-01-12" "2018-11-04"
[100] "2018-11-04" "2018-11-04" "2029-08-19" "2029-08-19" "2029-08-19" "2029-08-19" "2027-04-03" "2027-04-03" "2027-04-03"
[109] "2027-04-03" "2021-03-27" "2021-03-27" "2021-03-27" "2021-03-27" "2021-03-27" "2030-07-04" "2030-07-04" "2030-07-04"
[118] "2030-07-04" "2030-07-04" "2023-06-08" "2023-06-08" "2023-06-08" "2023-06-08" "2029-05-02" "2029-05-02" "2029-05-02"
[127] "2029-05-02" "2029-05-02" "2023-12-20" "2023-12-20" "2023-12-20" "2023-12-20" "2037-05-25" "2037-05-25" "2037-05-25"
[136] "2037-05-25" "2037-05-25" "2025-04-11" "2025-04-11" "2025-04-11" "2025-04-11" "2025-04-11" "2032-08-12" "2032-08-12"
[145] "2032-08-12" "2032-08-12" "2024-08-16" "2024-08-16" "2024-08-16" "2024-08-16" "2043-09-17" "2043-09-17" "2043-09-17"
[154] "2043-09-17" "2028-09-12" "2028-09-12" "2028-09-12" "2028-09-12" "2036-08-18" "2036-08-18" "2036-08-18" "2036-08-18"
[163] "2018-07-16" "2018-07-16" "2032-11-10" "2032-11-10" "2032-11-10" "2032-11-10" "2032-05-18" "2032-05-18" "2032-05-18"
[172] "2032-05-18" "2023-05-08" "2023-05-08" "2023-05-08" "2023-05-08" "2020-11-02" "2020-11-02" "2020-11-02" "2020-11-02"
[181] "2031-12-03" "2031-12-03" "2031-12-03" "2031-12-03" "2030-06-13" "2030-06-13" "2030-06-13" "2030-06-13" "2019-06-16"
[190] "2019-06-16" "2019-06-16" "2019-06-16" "2042-04-04" "2042-04-04" "2042-04-04" "2042-04-04" "2016-06-24" "2016-06-24"
[199] "2031-01-12" "2031-01-12" "2031-01-12" "2031-01-12" "2024-03-01" "2024-03-01" "2024-03-01" "2024-03-01" "2023-07-21"
[208] "2023-07-21" "2023-07-21" "2023-07-21" "2021-01-06" "2021-01-06" "2021-01-06" "2021-01-06" "2019-04-21" "2019-04-21"
[217] "2019-04-21" "2019-04-21" "2019-07-03" "2019-07-03" "2019-07-03" "2019-07-03" "2022-09-21" "2022-09-21" "2022-09-21"
[226] "2022-09-21" "2022-09-21" "2024-08-26" "2024-08-26" "2024-08-26" "2024-08-26" "2032-08-16" "2032-08-16" "2032-08-16"
[235] "2032-08-16" "2032-08-16" "2031-09-07" "2031-09-07" "2031-09-07" "2031-09-07" "2032-05-18" "2032-05-18" "2032-05-18"
[244] "2032-05-18" "2023-11-08" "2023-11-08" "2023-11-08" "2023-11-08" "2023-03-15" "2023-03-15" "2023-03-15" "2023-03-15"
[253] "2032-01-28" "2032-01-28" "2032-01-28" "2032-01-28" "2018-01-10" "2018-01-10" "2022-02-14" "2022-02-14" "2022-02-14"
[262] "2022-02-14" "2025-05-31" "2025-05-31" "2025-05-31" "2025-05-31" "2020-03-06" "2020-03-06" "2020-03-06" "2020-03-06"
[271] "2020-03-06" "2019-09-09" "2019-09-09" "2019-09-09" "2019-09-09" "2028-12-30" "2028-12-30" "2028-12-30" "2028-12-30"
[280] "2032-09-22" "2032-09-22" "2032-09-22" "2032-09-22" "2040-05-30" "2040-05-30" "2040-05-30" "2040-05-30" "2032-06-28"
[289] "2032-06-28" "2032-06-28" "2032-06-28" "2036-09-23" "2036-09-23" "2036-09-23" "2036-09-23" "2031-01-22" "2031-01-22"
[298] "2031-01-22" "2031-01-22" "2029-02-11" "2029-02-11" "2029-02-11" "2029-02-11" "2032-02-27" "2032-02-27" "2032-02-27"
[307] "2032-02-27" "2022-05-28" "2022-05-28" "2022-05-28" "2022-05-28" "2028-06-27" "2028-06-27" "2028-06-27" "2028-06-27"
[316] "2018-08-02" "2018-08-02" "2018-08-02" "2018-08-02" "2032-06-08" "2032-06-08" "2032-06-08" "2032-06-08" "2029-04-29"
[325] "2029-04-29" "2029-04-29" "2029-04-29" "2033-02-17" "2033-02-17" "2033-02-17" "2033-02-17" "2028-01-17" "2028-01-17"
[334] "2028-01-17" "2028-01-17" "2020-04-03" "2020-04-03" "2020-04-03" "2020-04-03" "2026-07-05" "2026-07-05" "2026-07-05"
[343] "2026-07-05" "2039-04-03" "2039-04-03" "2039-04-03" "2033-06-25" "2033-06-25" "2040-12-30" "2040-12-30" "2040-12-30"
[352] "2034-06-29" "2034-06-29" "2034-06-29" "2034-06-29" "2022-01-20" "2022-01-20" "2022-01-20" "2022-01-20" "2029-11-07"
[361] "2029-11-07" "2029-11-07" "2029-11-07" "2023-05-03" "2023-05-03" "2023-05-03" "2023-05-03" "2026-07-14" "2026-07-14"
[370] "2026-07-14" "2026-07-14" "2029-08-31" "2029-08-31" "2029-08-31" "2029-08-31" "2023-05-16" "2023-05-16" "2023-05-16"
[379] "2023-05-16" "2028-02-04" "2028-02-04" "2028-02-04" "2028-02-04" "2029-08-09" "2029-08-09" "2029-08-09" "2029-08-09"
[388] "2027-04-15" "2027-04-15" "2027-04-15" "2027-04-15" "2027-08-29" "2027-08-29" "2027-08-29" "2027-08-29" "2022-06-28"
[397] "2022-06-28" "2022-06-28" "2022-06-28" "2017-10-14" "2017-10-14" "2017-10-14" "2017-10-14" "2029-12-16" "2029-12-16"
[406] "2029-12-16" "2029-12-16" "2029-11-30" "2029-11-30" "2029-11-30" "2029-11-30" "2036-01-07" "2036-01-07" "2036-01-07"
[415] "2036-01-07" "2018-06-12" "2018-06-12" "2018-06-12" "2018-06-12" "2025-04-16" "2025-04-16" "2025-04-16" "2025-04-16"
[424] "2040-08-08" "2040-08-08" "2040-08-08" "2040-08-08" "2025-01-11" "2025-01-11" "2025-01-11" "2025-01-11" "2032-07-16"
[433] "2032-07-16" "2032-07-16" "2032-07-16" "2020-09-14" "2020-09-14" "2020-09-14" "2020-09-14" "2029-09-07" "2029-09-07"
[442] "2029-09-07" "2029-09-07" "2027-04-05" "2027-04-05" "2027-04-05" "2027-04-05" "2029-06-15" "2029-06-15" "2029-06-15"
[451] "2029-06-15" "2022-11-03" "2022-11-03" "2022-11-03" "2022-11-03" "2024-04-22" "2024-04-22" "2024-04-22" "2024-04-22"
[460] "2021-09-22" "2021-09-22" "2021-09-22" "2021-09-22" "2030-01-11" "2030-01-11" "2030-01-11" "2030-01-11" "2039-12-03"
[469] "2039-12-03" "2039-12-03" "2039-12-03" "2036-03-16" "2036-03-16" "2036-03-16" "2036-03-16" "2021-08-08" "2021-08-08"
[478] "2021-08-08" "2021-08-08" "2030-01-29" "2030-01-29" "2030-01-29" "2030-01-29" "2030-01-29" "2035-02-04" "2035-02-04"
[487] "2035-02-04" "2035-02-04" "2035-06-01" "2035-06-01" "2035-06-01" "2035-06-01" "2017-08-15" "2017-08-15" "2017-08-15"
[496] "2017-08-15" "2039-08-18" "2039-08-18" "2039-08-18" "2039-08-18" "2030-06-09" "2030-06-09" "2030-06-09" "2030-06-09"
[505] "2040-10-20" "2040-10-20" "2040-10-20" "2040-10-20" "2039-06-22" "2039-06-22" "2039-06-22" "2039-06-22" "2027-07-06"
[514] "2027-07-06" "2027-07-06" "2027-07-06" "2038-09-27" "2038-09-27" "2038-09-27" "2038-09-27" "2026-12-04" "2026-12-04"
[523] "2026-12-04" "2026-12-04" "2024-04-30" "2024-04-30" "2024-04-30" "2024-04-30" "2035-07-13" "2035-07-13" "2035-07-13"
[532] "2035-07-13" "2039-04-08" "2039-04-08" "2039-04-08" "2039-04-08" "2035-07-18" "2035-07-18" "2035-07-18" "2035-07-18"
[541] "2030-09-15" "2030-09-15" "2030-09-15" "2030-09-15" "2038-10-24" "2038-10-24" "2038-10-24" "2038-10-24" "2024-03-19"
[550] "2024-03-19" "2024-03-19" "2024-03-19" "2032-02-26" "2032-02-26" "2032-02-26" "2032-02-26" "2036-03-19" "2036-03-19"
[559] "2036-03-19" "2036-03-19" "2032-05-14" "2032-05-14" "2032-05-14" "2020-08-05" "2020-08-05" "2020-08-05" "2031-09-22"
[568] "2031-09-22" "2031-09-22" "2031-09-22" "2031-04-24" "2031-04-24" "2031-04-24" "2031-04-24" "2024-02-22" "2024-02-22"
[577] "2024-02-22" "2024-02-22" "2024-02-22" "2030-10-16" "2030-10-16" "2030-10-16" "2030-10-16" "2033-09-09" "2033-09-09"
[586] "2033-09-09" "2033-09-09" "2046-12-12" "2046-12-12" "2046-12-12" "2046-12-12" "2023-07-17" "2023-07-17" "2023-07-17"
[595] "2023-07-17" "2023-04-17" "2023-04-17" "2023-04-17" "2023-04-17" "2022-11-11" "2022-11-11" "2022-11-11" "2018-09-04"
[604] "2018-09-04" "2018-09-04" "2027-01-28" "2027-01-28" "2027-01-28" "2027-01-28" "2029-04-17" "2029-04-17" "2029-04-17"
[613] "2029-04-17" "2029-04-17" "2022-01-19" "2022-01-19" "2022-01-19" "2022-01-19" "2025-04-11" "2025-04-11" "2025-04-11"
[622] "2025-04-11" "2036-06-05" "2036-06-05" "2036-06-05" "2036-06-05" "2038-11-17" "2038-11-17" "2038-11-17" "2038-11-17"
[631] "2028-11-30" "2028-11-30" "2028-11-30" "2028-11-30" "2033-03-02" "2033-03-02" "2033-03-02" "2033-03-02" "2033-03-02"
[640] "2031-04-02" "2031-04-02" "2031-04-02" "2031-04-02" "2034-05-02" "2034-05-02" "2034-05-02" "2021-11-11" "2021-11-11"
[649] "2021-11-11" "2021-11-11" "2026-03-18" "2026-03-18" "2026-03-18" "2026-03-18" "2024-04-25" "2024-04-25" "2024-04-25"
[658] "2024-04-25" "2026-03-20" "2026-03-20" "2026-03-12" "2026-03-12" "2026-03-12" "2026-03-12" "2048-07-09" "2048-07-09"
[667] "2048-07-09" "2048-07-09" "2028-05-12" "2028-05-12" "2028-05-12" "2028-05-12" "2026-12-25" "2026-12-25" "2026-12-25"
[676] "2026-12-25" "2033-01-18" "2033-01-18" "2033-01-18" "2033-01-18" "2031-12-07" "2031-12-07" "2031-12-07" "2031-12-07"
[685] "2019-06-17" "2019-06-17" "2019-06-17" "2019-06-17" "2028-09-08" "2028-09-08" "2028-09-08" "2028-09-08" "2039-09-29"
[694] "2039-09-29" "2039-09-29" "2039-09-29" "2037-08-01" "2037-08-01" "2037-08-01" "2037-08-01" "2029-04-18" "2029-04-18"
[703] "2029-04-18" "2029-04-18" "2036-06-22" "2036-06-22" "2036-06-22" "2036-06-22" "2028-12-13" "2028-12-13" "2028-12-13"
[712] "2028-12-13" "2024-04-19" "2024-04-19" "2024-04-19" "2033-10-03" "2033-10-03" "2023-06-05" "2023-06-05" "2023-06-05"
[721] "2023-06-05" "2038-09-21" "2038-09-21" "2038-09-21" "2038-09-21" "2032-03-26" "2032-03-26" "2032-03-26" "2032-03-26"
[730] "2033-06-28" "2033-06-28" "2033-06-28" "2033-06-28" "2034-03-05" "2034-03-05" "2034-03-05" "2034-03-05" "2029-11-24"
[739] "2029-11-24" "2029-11-24" "2029-11-24" "2030-05-31" "2030-05-31" "2030-05-31" "2030-12-31" "2030-12-31" "2030-12-31"
[748] "2030-12-31" "2029-09-21" "2029-09-21" "2029-09-21" "2017-10-22" "2017-10-22" "2017-10-22" "2037-02-15" "2037-02-15"
[757] "2037-02-15" "2037-02-15" "2032-09-19" "2032-09-19" "2032-09-19" "2032-09-19" "2029-03-10" "2029-03-10" "2029-03-10"
[766] "2029-12-23" "2029-12-23" "2029-12-23" "2029-12-23" "2031-03-07" "2031-03-07" "2031-03-07" "2031-03-07" "2030-03-27"
[775] "2030-03-27" "2030-03-27" "2030-03-27" "2031-07-13" "2031-07-13" "2031-07-13" "2031-07-13" "2025-12-08" "2025-12-08"
[784] "2025-12-08" "2025-12-08" "2029-03-17" "2029-03-17" "2029-03-17" "2038-12-20" "2038-12-20" "2038-12-20" "2037-10-03"
[793] "2037-10-03" "2037-10-03" "2037-10-03" "2033-03-15" "2033-03-15" "2033-03-15" "2033-03-15" "2034-08-10" "2034-08-10"
[802] "2034-08-10" "2034-08-10" "2021-12-24" "2021-12-24" "2021-12-24" "2028-03-28" "2028-03-28" "2028-03-28" "2028-04-09"
[811] "2028-04-09" "2028-04-09"
> dt1=strptime(as.character(dat$BIRTHDT),'%d-%b-%y'); dt1$year=dt1$year-100; as.POSIXct(dt1)
[1] "1922-07-14 BST" "1922-07-14 BST" "1922-07-14 BST" "1922-07-14 BST" "1921-03-23 GMT" "1921-03-23 GMT" "1921-03-23 GMT"
[8] "1927-08-27 BST" "1927-08-27 BST" "1927-08-27 BST" "1927-08-27 BST" "1940-04-05 BST" "1940-04-05 BST" "1940-04-05 BST"
[15] "1940-04-05 BST" "1923-12-15 GMT" "1923-12-15 GMT" "1923-12-15 GMT" "1923-12-15 GMT" "1917-08-19 BST" "1917-08-19 BST"
[22] "1917-08-19 BST" "1917-08-19 BST" "1917-08-31 BST" "1917-08-31 BST" "1917-08-31 BST" "1917-08-31 BST" "1931-05-12 BST"
[29] "1931-05-12 BST" "1931-05-12 BST" "1931-05-12 BST" "1931-05-07 BST" "1931-05-07 BST" "1931-05-07 BST" "1931-05-07 BST"
[36] "1926-12-31 GMT" "1926-12-31 GMT" "1926-12-31 GMT" "1926-12-31 GMT" "1937-08-20 BST" "1937-08-20 BST" "1937-08-20 BST"
[43] "1937-08-20 BST" "1933-12-08 GMT" "1933-12-08 GMT" "1933-12-08 GMT" "1933-12-08 GMT" "1938-07-17 BST" "1938-07-17 BST"
[50] "1938-07-17 BST" "1938-07-17 BST" "1920-10-09 BST" "1920-10-09 BST" "1920-10-09 BST" "1920-10-09 BST" "1925-04-29 BST"
[57] "1925-04-29 BST" "1925-04-29 BST" "1925-04-29 BST" "1924-07-03 BST" "1924-07-03 BST" "1924-07-03 BST" "1924-07-03 BST"
[64] "1930-09-21 BST" "1930-09-21 BST" "1930-09-21 BST" "1930-09-21 BST" "1923-08-03 BST" "1923-08-03 BST" "1923-08-03 BST"
[71] "1923-08-03 BST" "1924-05-10 BST" "1924-05-10 BST" "1924-05-10 BST" "1924-05-10 BST" "1938-05-31 BST" "1938-05-31 BST"
[78] "1938-05-31 BST" "1938-05-31 BST" "1928-08-23 BST" "1928-08-23 BST" "1928-08-23 BST" "1928-08-23 BST" "1931-11-19 GMT"
[85] "1931-11-19 GMT" "1922-12-12 GMT" "1922-12-12 GMT" "1922-12-12 GMT" "1922-12-12 GMT" "1923-09-14 BST" "1923-09-14 BST"
[92] "1923-09-14 BST" "1923-09-14 BST" "1921-01-12 GMT" "1921-01-12 GMT" "1921-01-12 GMT" "1921-01-12 GMT" "1921-01-12 GMT"
[99] "1918-11-04 GMT" "1918-11-04 GMT" "1918-11-04 GMT" "1929-08-19 BST" "1929-08-19 BST" "1929-08-19 BST" "1929-08-19 BST"
[106] NA NA NA NA "1921-03-27 GMT" "1921-03-27 GMT" "1921-03-27 GMT"
[113] "1921-03-27 GMT" "1921-03-27 GMT" "1930-07-04 BST" "1930-07-04 BST" "1930-07-04 BST" "1930-07-04 BST" "1930-07-04 BST"
[120] "1923-06-08 BST" "1923-06-08 BST" "1923-06-08 BST" "1923-06-08 BST" "1929-05-02 BST" "1929-05-02 BST" "1929-05-02 BST"
[127] "1929-05-02 BST" "1929-05-02 BST" "1923-12-20 GMT" "1923-12-20 GMT" "1923-12-20 GMT" "1923-12-20 GMT" "1937-05-25 BST"
[134] "1937-05-25 BST" "1937-05-25 BST" "1937-05-25 BST" "1937-05-25 BST" NA NA NA
[141] NA NA "1932-08-12 BST" "1932-08-12 BST" "1932-08-12 BST" "1932-08-12 BST" "1924-08-16 BST"
[148] "1924-08-16 BST" "1924-08-16 BST" "1924-08-16 BST" "1943-09-17 BST" "1943-09-17 BST" "1943-09-17 BST" "1943-09-17 BST"
[155] "1928-09-12 BST" "1928-09-12 BST" "1928-09-12 BST" "1928-09-12 BST" "1936-08-18 BST" "1936-08-18 BST" "1936-08-18 BST"
[162] "1936-08-18 BST" "1918-07-16 BST" "1918-07-16 BST" "1932-11-10 GMT" "1932-11-10 GMT" "1932-11-10 GMT" "1932-11-10 GMT"
[169] "1932-05-18 BST" "1932-05-18 BST" "1932-05-18 BST" "1932-05-18 BST" "1923-05-08 BST" "1923-05-08 BST" "1923-05-08 BST"
[176] "1923-05-08 BST" "1920-11-02 GMT" "1920-11-02 GMT" "1920-11-02 GMT" "1920-11-02 GMT" "1931-12-03 GMT" "1931-12-03 GMT"
[183] "1931-12-03 GMT" "1931-12-03 GMT" "1930-06-13 BST" "1930-06-13 BST" "1930-06-13 BST" "1930-06-13 BST" "1919-06-16 BST"
[190] "1919-06-16 BST" "1919-06-16 BST" "1919-06-16 BST" "1942-04-04 BST" "1942-04-04 BST" "1942-04-04 BST" "1942-04-04 BST"
[197] "1916-06-24 BST" "1916-06-24 BST" "1931-01-12 GMT" "1931-01-12 GMT" "1931-01-12 GMT" "1931-01-12 GMT" "1924-03-01 GMT"
[204] "1924-03-01 GMT" "1924-03-01 GMT" "1924-03-01 GMT" "1923-07-21 BST" "1923-07-21 BST" "1923-07-21 BST" "1923-07-21 BST"
[211] "1921-01-06 GMT" "1921-01-06 GMT" "1921-01-06 GMT" "1921-01-06 GMT" "1919-04-21 BST" "1919-04-21 BST" "1919-04-21 BST"
[218] "1919-04-21 BST" "1919-07-03 BST" "1919-07-03 BST" "1919-07-03 BST" "1919-07-03 BST" "1922-09-21 BST" "1922-09-21 BST"
[225] "1922-09-21 BST" "1922-09-21 BST" "1922-09-21 BST" "1924-08-26 BST" "1924-08-26 BST" "1924-08-26 BST" "1924-08-26 BST"
[232] "1932-08-16 BST" "1932-08-16 BST" "1932-08-16 BST" "1932-08-16 BST" "1932-08-16 BST" "1931-09-07 BST" "1931-09-07 BST"
[239] "1931-09-07 BST" "1931-09-07 BST" "1932-05-18 BST" "1932-05-18 BST" "1932-05-18 BST" "1932-05-18 BST" "1923-11-08 GMT"
[246] "1923-11-08 GMT" "1923-11-08 GMT" "1923-11-08 GMT" "1923-03-15 GMT" "1923-03-15 GMT" "1923-03-15 GMT" "1923-03-15 GMT"
[253] "1932-01-28 GMT" "1932-01-28 GMT" "1932-01-28 GMT" "1932-01-28 GMT" "1918-01-10 GMT" "1918-01-10 GMT" "1922-02-14 GMT"
[260] "1922-02-14 GMT" "1922-02-14 GMT" "1922-02-14 GMT" "1925-05-31 BST" "1925-05-31 BST" "1925-05-31 BST" "1925-05-31 BST"
[267] "1920-03-06 GMT" "1920-03-06 GMT" "1920-03-06 GMT" "1920-03-06 GMT" "1920-03-06 GMT" "1919-09-09 BST" "1919-09-09 BST"
[274] "1919-09-09 BST" "1919-09-09 BST" "1928-12-30 GMT" "1928-12-30 GMT" "1928-12-30 GMT" "1928-12-30 GMT" "1932-09-22 BST"
[281] "1932-09-22 BST" "1932-09-22 BST" "1932-09-22 BST" "1940-05-30 BST" "1940-05-30 BST" "1940-05-30 BST" "1940-05-30 BST"
[288] "1932-06-28 BST" "1932-06-28 BST" "1932-06-28 BST" "1932-06-28 BST" "1936-09-23 BST" "1936-09-23 BST" "1936-09-23 BST"
[295] "1936-09-23 BST" "1931-01-22 GMT" "1931-01-22 GMT" "1931-01-22 GMT" "1931-01-22 GMT" "1929-02-11 GMT" "1929-02-11 GMT"
[302] "1929-02-11 GMT" "1929-02-11 GMT" "1932-02-27 GMT" "1932-02-27 GMT" "1932-02-27 GMT" "1932-02-27 GMT" "1922-05-28 BST"
[309] "1922-05-28 BST" "1922-05-28 BST" "1922-05-28 BST" "1928-06-27 BST" "1928-06-27 BST" "1928-06-27 BST" "1928-06-27 BST"
[316] "1918-08-02 BST" "1918-08-02 BST" "1918-08-02 BST" "1918-08-02 BST" "1932-06-08 BST" "1932-06-08 BST" "1932-06-08 BST"
[323] "1932-06-08 BST" "1929-04-29 BST" "1929-04-29 BST" "1929-04-29 BST" "1929-04-29 BST" "1933-02-17 GMT" "1933-02-17 GMT"
[330] "1933-02-17 GMT" "1933-02-17 GMT" "1928-01-17 GMT" "1928-01-17 GMT" "1928-01-17 GMT" "1928-01-17 GMT" "1920-04-03 BST"
[337] "1920-04-03 BST" "1920-04-03 BST" "1920-04-03 BST" "1926-07-05 BST" "1926-07-05 BST" "1926-07-05 BST" "1926-07-05 BST"
[344] NA NA NA "1933-06-25 BST" "1933-06-25 BST" NA NA
[351] NA "1934-06-29 BST" "1934-06-29 BST" "1934-06-29 BST" "1934-06-29 BST" "1922-01-20 GMT" "1922-01-20 GMT"
[358] "1922-01-20 GMT" "1922-01-20 GMT" "1929-11-07 GMT" "1929-11-07 GMT" "1929-11-07 GMT" "1929-11-07 GMT" "1923-05-03 BST"
[365] "1923-05-03 BST" "1923-05-03 BST" "1923-05-03 BST" "1926-07-14 BST" "1926-07-14 BST" "1926-07-14 BST" "1926-07-14 BST"
[372] "1929-08-31 BST" "1929-08-31 BST" "1929-08-31 BST" "1929-08-31 BST" "1923-05-16 BST" "1923-05-16 BST" "1923-05-16 BST"
[379] "1923-05-16 BST" "1928-02-04 GMT" "1928-02-04 GMT" "1928-02-04 GMT" "1928-02-04 GMT" "1929-08-09 BST" "1929-08-09 BST"
[386] "1929-08-09 BST" "1929-08-09 BST" "1927-04-15 BST" "1927-04-15 BST" "1927-04-15 BST" "1927-04-15 BST" "1927-08-29 BST"
[393] "1927-08-29 BST" "1927-08-29 BST" "1927-08-29 BST" "1922-06-28 BST" "1922-06-28 BST" "1922-06-28 BST" "1922-06-28 BST"
[400] NA NA NA NA "1929-12-16 GMT" "1929-12-16 GMT" "1929-12-16 GMT"
[407] "1929-12-16 GMT" "1929-11-30 GMT" "1929-11-30 GMT" "1929-11-30 GMT" "1929-11-30 GMT" "1936-01-07 GMT" "1936-01-07 GMT"
[414] "1936-01-07 GMT" "1936-01-07 GMT" "1918-06-12 BST" "1918-06-12 BST" "1918-06-12 BST" "1918-06-12 BST" NA
[421] NA NA NA "1940-08-08 BST" "1940-08-08 BST" "1940-08-08 BST" "1940-08-08 BST"
[428] "1925-01-11 GMT" "1925-01-11 GMT" "1925-01-11 GMT" "1925-01-11 GMT" "1932-07-16 BST" "1932-07-16 BST" "1932-07-16 BST"
[435] "1932-07-16 BST" "1920-09-14 BST" "1920-09-14 BST" "1920-09-14 BST" "1920-09-14 BST" "1929-09-07 BST" "1929-09-07 BST"
[442] "1929-09-07 BST" "1929-09-07 BST" NA NA NA NA "1929-06-15 BST"
[449] "1929-06-15 BST" "1929-06-15 BST" "1929-06-15 BST" "1922-11-03 GMT" "1922-11-03 GMT" "1922-11-03 GMT" "1922-11-03 GMT"
[456] "1924-04-22 BST" "1924-04-22 BST" "1924-04-22 BST" "1924-04-22 BST" "1921-09-22 BST" "1921-09-22 BST" "1921-09-22 BST"
[463] "1921-09-22 BST" "1930-01-11 GMT" "1930-01-11 GMT" "1930-01-11 GMT" "1930-01-11 GMT" "1939-12-03 GMT" "1939-12-03 GMT"
[470] "1939-12-03 GMT" "1939-12-03 GMT" "1936-03-16 GMT" "1936-03-16 GMT" "1936-03-16 GMT" "1936-03-16 GMT" "1921-08-08 BST"
[477] "1921-08-08 BST" "1921-08-08 BST" "1921-08-08 BST" "1930-01-29 GMT" "1930-01-29 GMT" "1930-01-29 GMT" "1930-01-29 GMT"
[484] "1930-01-29 GMT" "1935-02-04 GMT" "1935-02-04 GMT" "1935-02-04 GMT" "1935-02-04 GMT" "1935-06-01 BST" "1935-06-01 BST"
[491] "1935-06-01 BST" "1935-06-01 BST" "1917-08-15 BST" "1917-08-15 BST" "1917-08-15 BST" "1917-08-15 BST" "1939-08-18 BST"
[498] "1939-08-18 BST" "1939-08-18 BST" "1939-08-18 BST" "1930-06-09 BST" "1930-06-09 BST" "1930-06-09 BST" "1930-06-09 BST"
[505] "1940-10-20 BST" "1940-10-20 BST" "1940-10-20 BST" "1940-10-20 BST" "1939-06-22 BST" "1939-06-22 BST" "1939-06-22 BST"
[512] "1939-06-22 BST" "1927-07-06 BST" "1927-07-06 BST" "1927-07-06 BST" "1927-07-06 BST" "1938-09-27 BST" "1938-09-27 BST"
[519] "1938-09-27 BST" "1938-09-27 BST" "1926-12-04 GMT" "1926-12-04 GMT" "1926-12-04 GMT" "1926-12-04 GMT" "1924-04-30 BST"
[526] "1924-04-30 BST" "1924-04-30 BST" "1924-04-30 BST" "1935-07-13 BST" "1935-07-13 BST" "1935-07-13 BST" "1935-07-13 BST"
[533] NA NA NA NA "1935-07-18 BST" "1935-07-18 BST" "1935-07-18 BST"
[540] "1935-07-18 BST" "1930-09-15 BST" "1930-09-15 BST" "1930-09-15 BST" "1930-09-15 BST" NA NA
[547] NA NA "1924-03-19 GMT" "1924-03-19 GMT" "1924-03-19 GMT" "1924-03-19 GMT" "1932-02-26 GMT"
[554] "1932-02-26 GMT" "1932-02-26 GMT" "1932-02-26 GMT" "1936-03-19 GMT" "1936-03-19 GMT" "1936-03-19 GMT" "1936-03-19 GMT"
[561] "1932-05-14 BST" "1932-05-14 BST" "1932-05-14 BST" "1920-08-05 BST" "1920-08-05 BST" "1920-08-05 BST" "1931-09-22 BST"
[568] "1931-09-22 BST" "1931-09-22 BST" "1931-09-22 BST" "1931-04-24 BST" "1931-04-24 BST" "1931-04-24 BST" "1931-04-24 BST"
[575] "1924-02-22 GMT" "1924-02-22 GMT" "1924-02-22 GMT" "1924-02-22 GMT" "1924-02-22 GMT" NA NA
[582] NA NA "1933-09-09 BST" "1933-09-09 BST" "1933-09-09 BST" "1933-09-09 BST" "1946-12-12 GMT"
[589] "1946-12-12 GMT" "1946-12-12 GMT" "1946-12-12 GMT" "1923-07-17 BST" "1923-07-17 BST" "1923-07-17 BST" "1923-07-17 BST"
[596] NA NA NA NA "1922-11-11 GMT" "1922-11-11 GMT" "1922-11-11 GMT"
[603] "1918-09-04 BST" "1918-09-04 BST" "1918-09-04 BST" "1927-01-28 GMT" "1927-01-28 GMT" "1927-01-28 GMT" "1927-01-28 GMT"
[610] NA NA NA NA NA "1922-01-19 GMT" "1922-01-19 GMT"
[617] "1922-01-19 GMT" "1922-01-19 GMT" NA NA NA NA "1936-06-05 BST"
[624] "1936-06-05 BST" "1936-06-05 BST" "1936-06-05 BST" "1938-11-17 GMT" "1938-11-17 GMT" "1938-11-17 GMT" "1938-11-17 GMT"
[631] "1928-11-30 GMT" "1928-11-30 GMT" "1928-11-30 GMT" "1928-11-30 GMT" "1933-03-02 GMT" "1933-03-02 GMT" "1933-03-02 GMT"
[638] "1933-03-02 GMT" "1933-03-02 GMT" NA NA NA NA "1934-05-02 BST"
[645] "1934-05-02 BST" "1934-05-02 BST" "1921-11-11 GMT" "1921-11-11 GMT" "1921-11-11 GMT" "1921-11-11 GMT" "1926-03-18 GMT"
[652] "1926-03-18 GMT" "1926-03-18 GMT" "1926-03-18 GMT" "1924-04-25 BST" "1924-04-25 BST" "1924-04-25 BST" "1924-04-25 BST"
[659] "1926-03-20 GMT" "1926-03-20 GMT" "1926-03-12 GMT" "1926-03-12 GMT" "1926-03-12 GMT" "1926-03-12 GMT" "1948-07-09 BST"
[666] "1948-07-09 BST" "1948-07-09 BST" "1948-07-09 BST" "1928-05-12 BST" "1928-05-12 BST" "1928-05-12 BST" "1928-05-12 BST"
[673] "1926-12-25 GMT" "1926-12-25 GMT" "1926-12-25 GMT" "1926-12-25 GMT" "1933-01-18 GMT" "1933-01-18 GMT" "1933-01-18 GMT"
[680] "1933-01-18 GMT" "1931-12-07 GMT" "1931-12-07 GMT" "1931-12-07 GMT" "1931-12-07 GMT" "1919-06-17 BST" "1919-06-17 BST"
[687] "1919-06-17 BST" "1919-06-17 BST" "1928-09-08 BST" "1928-09-08 BST" "1928-09-08 BST" "1928-09-08 BST" "1939-09-29 BST"
[694] "1939-09-29 BST" "1939-09-29 BST" "1939-09-29 BST" "1937-08-01 BST" "1937-08-01 BST" "1937-08-01 BST" "1937-08-01 BST"
[701] NA NA NA NA "1936-06-22 BST" "1936-06-22 BST" "1936-06-22 BST"
[708] "1936-06-22 BST" "1928-12-13 GMT" "1928-12-13 GMT" "1928-12-13 GMT" "1928-12-13 GMT" "1924-04-19 BST" "1924-04-19 BST"
[715] "1924-04-19 BST" "1933-10-03 BST" "1933-10-03 BST" "1923-06-05 BST" "1923-06-05 BST" "1923-06-05 BST" "1923-06-05 BST"
[722] "1938-09-21 BST" "1938-09-21 BST" "1938-09-21 BST" "1938-09-21 BST" "1932-03-26 GMT" "1932-03-26 GMT" "1932-03-26 GMT"
[729] "1932-03-26 GMT" "1933-06-28 BST" "1933-06-28 BST" "1933-06-28 BST" "1933-06-28 BST" "1934-03-05 GMT" "1934-03-05 GMT"
[736] "1934-03-05 GMT" "1934-03-05 GMT" "1929-11-24 GMT" "1929-11-24 GMT" "1929-11-24 GMT" "1929-11-24 GMT" "1930-05-31 BST"
[743] "1930-05-31 BST" "1930-05-31 BST" "1930-12-31 GMT" "1930-12-31 GMT" "1930-12-31 GMT" "1930-12-31 GMT" "1929-09-21 BST"
[750] "1929-09-21 BST" "1929-09-21 BST" NA NA NA "1937-02-15 GMT" "1937-02-15 GMT"
[757] "1937-02-15 GMT" "1937-02-15 GMT" "1932-09-19 BST" "1932-09-19 BST" "1932-09-19 BST" "1932-09-19 BST" "1929-03-10 GMT"
[764] "1929-03-10 GMT" "1929-03-10 GMT" "1929-12-23 GMT" "1929-12-23 GMT" "1929-12-23 GMT" "1929-12-23 GMT" "1931-03-07 GMT"
[771] "1931-03-07 GMT" "1931-03-07 GMT" "1931-03-07 GMT" "1930-03-27 GMT" "1930-03-27 GMT" "1930-03-27 GMT" "1930-03-27 GMT"
[778] "1931-07-13 BST" "1931-07-13 BST" "1931-07-13 BST" "1931-07-13 BST" "1925-12-08 GMT" "1925-12-08 GMT" "1925-12-08 GMT"
[785] "1925-12-08 GMT" "1929-03-17 GMT" "1929-03-17 GMT" "1929-03-17 GMT" "1938-12-20 GMT" "1938-12-20 GMT" "1938-12-20 GMT"
[792] "1937-10-03 BST" "1937-10-03 BST" "1937-10-03 BST" "1937-10-03 BST" "1933-03-15 GMT" "1933-03-15 GMT" "1933-03-15 GMT"
[799] "1933-03-15 GMT" "1934-08-10 BST" "1934-08-10 BST" "1934-08-10 BST" "1934-08-10 BST" "1921-12-24 GMT" "1921-12-24 GMT"
[806] "1921-12-24 GMT" NA NA NA NA NA NA
If I look at POSIXct(dob) before subtracting 100 from dob$year, then there are no missing values. So, it does seem that the problem arises solely as a result of subtracting 100 from dob$year. But it is still not clear to me what this problem is. I must re-emphasise that subtracting 100 from dob$year does not result in any missing values:-
dt1$year
[1] 22 22 22 22 21 21 21 27 27 27 27 40 40 40 40 23 23 23 23 17 17 17 17 17 17 17 17 31 31 31 31 31 31 31 31 26 26 26 26
[40] 37 37 37 37 33 33 33 33 38 38 38 38 20 20 20 20 25 25 25 25 24 24 24 24 30 30 30 30 23 23 23 23 24 24 24 24 38 38 38
[79] 38 28 28 28 28 31 31 22 22 22 22 23 23 23 23 21 21 21 21 21 18 18 18 29 29 29 29 27 27 27 27 21 21 21 21 21 30 30 30
[118] 30 30 23 23 23 23 29 29 29 29 29 23 23 23 23 37 37 37 37 37 25 25 25 25 25 32 32 32 32 24 24 24 24 43 43 43 43 28 28
[157] 28 28 36 36 36 36 18 18 32 32 32 32 32 32 32 32 23 23 23 23 20 20 20 20 31 31 31 31 30 30 30 30 19 19 19 19 42 42 42
[196] 42 16 16 31 31 31 31 24 24 24 24 23 23 23 23 21 21 21 21 19 19 19 19 19 19 19 19 22 22 22 22 22 24 24 24 24 32 32 32
[235] 32 32 31 31 31 31 32 32 32 32 23 23 23 23 23 23 23 23 32 32 32 32 18 18 22 22 22 22 25 25 25 25 20 20 20 20 20 19 19
[274] 19 19 28 28 28 28 32 32 32 32 40 40 40 40 32 32 32 32 36 36 36 36 31 31 31 31 29 29 29 29 32 32 32 32 22 22 22 22 28
[313] 28 28 28 18 18 18 18 32 32 32 32 29 29 29 29 33 33 33 33 28 28 28 28 20 20 20 20 26 26 26 26 39 39 39 33 33 40 40 40
[352] 34 34 34 34 22 22 22 22 29 29 29 29 23 23 23 23 26 26 26 26 29 29 29 29 23 23 23 23 28 28 28 28 29 29 29 29 27 27 27
[391] 27 27 27 27 27 22 22 22 22 17 17 17 17 29 29 29 29 29 29 29 29 36 36 36 36 18 18 18 18 25 25 25 25 40 40 40 40 25 25
[430] 25 25 32 32 32 32 20 20 20 20 29 29 29 29 27 27 27 27 29 29 29 29 22 22 22 22 24 24 24 24 21 21 21 21 30 30 30 30 39
[469] 39 39 39 36 36 36 36 21 21 21 21 30 30 30 30 30 35 35 35 35 35 35 35 35 17 17 17 17 39 39 39 39 30 30 30 30 40 40 40
[508] 40 39 39 39 39 27 27 27 27 38 38 38 38 26 26 26 26 24 24 24 24 35 35 35 35 39 39 39 39 35 35 35 35 30 30 30 30 38 38
[547] 38 38 24 24 24 24 32 32 32 32 36 36 36 36 32 32 32 20 20 20 31 31 31 31 31 31 31 31 24 24 24 24 24 30 30 30 30 33 33
[586] 33 33 46 46 46 46 23 23 23 23 23 23 23 23 22 22 22 18 18 18 27 27 27 27 29 29 29 29 29 22 22 22 22 25 25 25 25 36 36
[625] 36 36 38 38 38 38 28 28 28 28 33 33 33 33 33 31 31 31 31 34 34 34 21 21 21 21 26 26 26 26 24 24 24 24 26 26 26 26 26
[664] 26 48 48 48 48 28 28 28 28 26 26 26 26 33 33 33 33 31 31 31 31 19 19 19 19 28 28 28 28 39 39 39 39 37 37 37 37 29 29
[703] 29 29 36 36 36 36 28 28 28 28 24 24 24 33 33 23 23 23 23 38 38 38 38 32 32 32 32 33 33 33 33 34 34 34 34 29 29 29 29
[742] 30 30 30 30 30 30 30 29 29 29 17 17 17 37 37 37 37 32 32 32 32 29 29 29 29 29 29 29 31 31 31 31 30 30 30 30 31 31 31
[781] 31 25 25 25 25 29 29 29 38 38 38 37 37 37 37 33 33 33 33 34 34 34 34 21 21 21 28 28 28 28 28 28
> str(dt1$year)
num [1:812] 22 22 22 22 21 21 21 27 27 27 ...
> table(is.na(dt1$year))
FALSE
812
> table(dt1$year)
16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 42 43 46 48
2 19 18 20 24 32 40 56 44 29 30 28 46 71 45 50 60 33 15 16 28 21 27 23 19 4 4 4 4
So, I'm still not really further forward. However, I suppose I could now work out a fix for my program, by difftime-ing dob with (19)99-12-31 and then difftime-ing (20)00-01-01 with sdate and adding the two difftimes. Still it irks me that I don't feel I understand nor can trust the results of strptime, if simply subtracting 100 from the year (which I suspect is a common procedure in many situations) does not yield reliable results.
Thanks for your help. Can anyone else illuminate this difficulty?
Jonathan Williams
PS I am running R 2.9.2 on a Windows XP machine.
________________________________________
From: Don MacQueen [macq at llnl.gov]
Sent: 23 February 2010 21:25
To: Jonathan Williams; r-help at r-project.org
Subject: Re: [R] Problem with strptime generating missing values where none appear to exist
What happens if you do all that NA checking on dob *before*
subtracting 100 from dob$year?
What happens if you use difftime() before subtracting the 100?
Do you get any NAs if you convert dob to POSIXct?
(these are just investigative ideas, obviously)
-Don
At 6:26 PM +0000 2/23/10, Jonathan Williams wrote:
>Dear R Helpers,
>
>I am having difficulty with strptime. I wish to find the differences between
>two vectors of times. I have apparently no difficulty to convert the vectors
>to the appropriate format using strptime. But, then difftime does not
>calculate all the differences.
>
>Here is the code and output:-
>
>dob=strptime(as.character(datx$BDT),'%d-%b-%y'); dob$year=dob$year-100
>sdate=strptime(as.character(datx$SDT),'%d-%b-%y')
>head(dob); head(sdate)
>[1] "1922-07-14" "1922-07-14" "1922-07-14" "1922-07-14" "1921-03-23"
>"1921-03-23"
>[1] "2001-08-27" "2001-08-27" "2001-08-27" "2001-08-27" "2001-08-20"
>"2001-08-20"
>> str(dob)
> POSIXlt[1:9], format: "1922-07-14" "1922-07-14" "1922-07-14" "1922-07-14"
>"1921-03-23" "1921-03-23" "1921-03-23" "1927-08-27" "1927-08-27"
>"1927-08-27" "1927-08-27" "1940-04-05" "1940-04-05" "1940-04-05"
>"1940-04-05" ...
>> str(sdate)
> POSIXlt[1:9], format: "2001-08-27" "2001-08-27" "2001-08-27" "2001-08-27"
>"2001-08-20" "2001-08-20" "2001-08-20" "2001-11-26" "2001-11-26"
>"2001-11-26" "2001-11-26" "2002-05-20" "2002-05-20" "2002-05-20"
>"2002-05-20" ...
>
>table(is.na(sdate))
>
>FALSE
> 812
>
>table(is.na(dob))
>
>FALSE TRUE
> 743 69
>But if I now look at each component of dob separately, none is missing
>
>for (i in 1:length(dob)) {print(names(dob)[i]);
>print(table(is.na(dob[[i]])))}
>
>[1] "sec"
>
>FALSE
> 812
>[1] "min"
>
>FALSE
> 812
>[1] "hour"
>
>FALSE
> 812
>[1] "mday"
>
>FALSE
> 812
>[1] "mon"
>
>FALSE
> 812
>[1] "year"
>
>FALSE
> 812
>[1] "wday"
>
>FALSE
> 812
>[1] "yday"
>
>FALSE
> 812
>[1] "isdst"
>
>FALSE
> 812
>
>Additionally, there are no NA values in any component of dob on direct
>visual inspection. For example, here is dob$mon
>dob$mon
>[1] 6 6 6 6 2 2 2 7 7 7 7 3 3 3 3 11 11 11 11 7 7 7 7 7
>7 7 7 4 4 4 4 4 4 4 4 11 11 11 11 7 7 7 7 11 11 11 11 6 6 6
>6 9 9 9 9 3 3 3 3 6 6 6 6 8 8 8 8 7 7 7 7 4 4 4 4 4
> [77] 4 4 4 7 7 7 7 10 10 11 11 11 11 8 8 8 8 0 0 0 0 0 10
>10 10 7 7 7 7 3 3 3 3 2 2 2 2 2 6 6 6 6 6 5 5 5 5 4
>4 4 4 4 11 11 11 11 4 4 4 4 4 3 3 3 3 3 7 7 7 7 7 7 7 7
>8 8
>[153] 8 8 8 8 8 8 7 7 7 7 6 6 10 10 10 10 4 4 4 4 4 4 4
>4 10 10 10 10 11 11 11 11 5 5 5 5 5 5 5 5 3 3 3 3 5 5 0 0 0
>0 2 2 2 2 6 6 6 6 0 0 0 0 3 3 3 3 6 6 6 6 8 8 8 8 8
>7
>[229] 7 7 7 7 7 7 7 7 8 8 8 8 4 4 4 4 10 10 10 10 2 2 2
>2 0 0 0 0 0 0 1 1 1 1 4 4 4 4 2 2 2 2 2 8 8 8 8 11 11
>11 11 8 8 8 8 4 4 4 4 5 5 5 5 8 8 8 8 0 0 0 0 1 1 1
>1 1
>[305] 1 1 1 4 4 4 4 5 5 5 5 7 7 7 7 5 5 5 5 3 3 3 3
>1 1 1 1 0 0 0 0 3 3 3 3 6 6 6 6 3 3 3 5 5 11 11 11 5 5
>5 5 0 0 0 0 10 10 10 10 4 4 4 4 6 6 6 6 7 7 7 7 4 4 4 4
>1
>[381] 1 1 1 7 7 7 7 3 3 3 3 7 7 7 7 5 5 5 5 9 9 9 9
>11 11 11 11 10 10 10 10 0 0 0 0 5 5 5 5 3 3 3 3 7 7 7 7 0
>0 0 0 6 6 6 6 8 8 8 8 8 8 8 8 3 3 3 3 5 5 5 5 10 10 10
>10 3
>[457] 3 3 3 8 8 8 8 0 0 0 0 11 11 11 11 2 2 2 2 7 7 7 7
>0 0 0 0 0 1 1 1 1 5 5 5 5 7 7 7 7 7 7 7 7 5 5 5 5 9
>9 9 9 5 5 5 5 6 6 6 6 8 8 8 8 11 11 11 11 3 3 3 3 6 6 6
>6
>[533] 3 3 3 3 6 6 6 6 8 8 8 8 9 9 9 9 2 2 2 2 1 1 1
>1 2 2 2 2 4 4 4 7 7 7 8 8 8 8 3 3 3 3 1 1 1 1 1 9 9
>9 9 8 8 8 8 11 11 11 11 6 6 6 6 3 3 3 3 10 10 10 8 8 8 0 0
>0
>[609] 0 3 3 3 3 3 0 0 0 0 3 3 3 3 5 5 5 5 10 10 10 10 10
>10 10 10 2 2 2 2 2 3 3 3 3 4 4 4 10 10 10 10 2 2 2 2 3 3
>3 3 2 2 2 2 2 2 6 6 6 6 4 4 4 4 11 11 11 11 0 0 0 0 11 11
>11 11
>[685] 5 5 5 5 8 8 8 8 8 8 8 8 7 7 7 7 3 3 3 3 5 5 5
>5 11 11 11 11 3 3 3 9 9 5 5 5 5 8 8 8 8 2 2 2 2 5 5 5 5
>2 2 2 2 10 10 10 10 4 4 4 11 11 11 11 8 8 8 9 9 9 1 1 1 1 8
>8
>[761] 8 8 2 2 2 11 11 11 11 2 2 2 2 2 2 2 2 6 6 6 6 11 11
>11 11 2 2 2 11 11 11 9 9 9 9 2 2 2 2 7 7 7 7 11 11 11 2 2
>2 3 3 3
>
>All the dob components are equally complete, including isdst.
>
>However, when I then try to compute difftime(sdate,dob), 69 values are
>missing:-
>Time differences in days
> [1] 28899.00 28899.00 28899.00 28899.00 29369.96 29369.96 29369.96
>27120.04 27120.04 27120.04 27120.04 22690.00 22690.00 22690.00 22690.00
>28905.00 28905.00 28905.00 28905.00 31207.04 31207.04 31207.04 31207.04
>31209.04 31209.04
> [26] 31209.04 31209.04 26323.00 26323.00 26323.00 26323.00 26338.00
>26338.00 26338.00 26338.00 27310.96 27310.96 27310.96 27310.96 23588.04
>23588.04 23588.04 23588.04 25255.00 25255.00 25255.00 25255.00 23752.00
>23752.00 23752.00
> [51] 23752.00 29607.04 29607.04 29607.04 29607.04 27993.04 27993.04
>27993.04 27993.04 28384.04 28384.04 28384.04 28384.04 26176.00 26176.00
>26176.00 26176.00 28986.04 28986.04 28986.04 28986.04 28689.04 28689.04
>28689.04 28689.04
> [76] 23722.00 23722.00 23722.00 23722.00 27353.00 27353.00 27353.00
>27353.00 26303.00 26303.00 28803.96 28803.96 28803.96 28803.96 28564.04
>28564.04 28564.04 28564.04 29826.96 29826.96 29826.96 29826.96 29826.96
>30410.00 30410.00
>[101] 30410.00 26490.04 26490.04 26490.04 26490.04 NA NA
>NA NA 29765.96 29765.96 29765.96 29765.96 29765.96 26325.00 26325.00
>26325.00 26325.00 26325.00 28824.00 28824.00 28824.00 28824.00 26808.00
>26808.00
>[126] 26808.00 26808.00 26808.00 28628.96 28628.96 28628.96 28628.96
>23807.00 23807.00 23807.00 23807.00 23807.00 NA NA NA
>NA NA 25668.04 25668.04 25668.04 25668.04 28654.04 28654.04 28654.04
>28654.04
>[151] 21711.04 21711.04 21711.04 21711.04 27167.04 27167.04 27167.04
>27167.04 24296.04 24296.04 24296.04 24296.04 30540.04 30540.04 25330.00
>25330.00 25330.00 25330.00 25579.00 25579.00 25579.00 25579.00 29127.04
>29127.04 29127.04
>[176] 29127.04 29896.96 29896.96 29896.96 29896.96 25992.00 25992.00
>25992.00 25992.00 26625.00 26625.00 26625.00 26625.00 30121.04 30121.04
>30121.04 30121.04 21801.04 21801.04 21801.04 21801.04 31274.04 31274.04
>25907.00 25907.00
>[201] 25907.00 25907.00 28516.00 28516.00 28516.00 28516.00 28943.00
>28943.00 28943.00 28943.00 29847.96 29847.96 29847.96 29847.96 30529.04
>30529.04 30529.04 30529.04 30527.04 30527.04 30527.04 30527.04 29434.00
>29434.00 29434.00
>[226] 29434.00 29434.00 28631.04 28631.04 28631.04 28631.04 25761.04
>25761.04 25761.04 25761.04 25761.04 26127.04 26127.04 26127.04 26127.04
>26027.00 26027.00 26027.00 26027.00 28987.00 28987.00 28987.00 28987.00
>29232.00 29232.00
>[251] 29232.00 29232.00 26109.96 26109.96 26109.96 26109.96 31339.00
>31339.00 29235.00 29235.00 29235.00 29235.00 28092.00 28092.00 28092.00
>28092.00 30209.00 30209.00 30209.00 30209.00 30209.00 30281.00 30281.00
>30281.00 30281.00
>[276] 26880.96 26880.96 26880.96 26880.96 25691.04 25691.04 25691.04
>25691.04 22938.04 22938.04 22938.04 22938.04 25878.00 25878.00 25878.00
>25878.00 24470.00 24470.00 24470.00 24470.00 26046.96 26046.96 26046.96
>26046.96 26763.96
>[301] 26763.96 26763.96 26763.96 25720.96 25720.96 25720.96 25720.96
>29214.00 29214.00 29214.00 29214.00 26992.00 26992.00 26992.00 26992.00
>30659.00 30659.00 30659.00 30659.00 25600.00 25600.00 25600.00 25600.00
>26842.00 26842.00
>[326] 26842.00 26842.00 25541.00 25541.00 25541.00 25541.00 27386.00
>27386.00 27386.00 27386.00 30302.04 30302.04 30302.04 30302.04 28059.00
>28059.00 28059.00 28059.00 NA NA NA 25657.00 25657.00
>NA NA
>[351] NA 24835.00 24835.00 24835.00 24835.00 29340.96 29340.96
>29340.96 29340.96 26473.96 26473.96 26473.96 26473.96 28873.00 28873.00
>28873.00 28873.00 27690.00 27690.00 27690.00 27690.00 26554.00 26554.00
>26554.00 26554.00
>[376] 28876.00 28876.00 28876.00 28876.00 27156.96 27156.96 27156.96
>27156.96 26577.00 26577.00 26577.00 26577.00 27471.00 27471.00 27471.00
>27471.00 27323.00 27323.00 27323.00 27323.00 29232.00 29232.00 29232.00
>29232.00 NA
>[401] NA NA NA 26523.96 26523.96 26523.96 26523.96
>26538.96 26538.96 26538.96 26538.96 24374.96 24374.96 24374.96 24374.96
>30798.00 30798.00 30798.00 30798.00 NA NA NA NA
>22775.04 22775.04
>[426] 22775.04 22775.04 28464.00 28464.00 28464.00 28464.00 25763.04
>25763.04 25763.04 25763.04 30114.04 30114.04 30114.04 30114.04 26864.04
>26864.04 26864.04 26864.04 NA NA NA NA 26945.04
>26945.04 26945.04
>[451] 26945.04 29528.96 29528.96 29528.96 29528.96 29058.04 29058.04
>29058.04 29058.04 29456.00 29456.00 29456.00 29456.00 26450.96 26450.96
>26450.96 26450.96 22837.96 22837.96 22837.96 22837.96 24222.96 24222.96
>24222.96 24222.96
>[476] 29592.00 29592.00 29592.00 29592.00 26573.00 26573.00 26573.00
>26573.00 26573.00 24811.00 24811.00 24811.00 24811.00 24834.00 24834.00
>24834.00 24834.00 31312.00 31312.00 31312.00 31312.00 23337.00 23337.00
>23337.00 23337.00
>[501] 26422.00 26422.00 26422.00 26422.00 22664.04 22664.04 22664.04
>22664.04 23192.04 23192.04 23192.04 23192.04 27557.04 27557.04 27557.04
>27557.04 23449.04 23449.04 23449.04 23449.04 27799.00 27799.00 27799.00
>27799.00 28747.04
>[526] 28747.04 28747.04 28747.04 24660.04 24660.04 24660.04 24660.04
>NA NA NA NA 24683.04 24683.04 24683.04 24683.04 26576.00
>26576.00 26576.00 26576.00 NA NA NA NA 28897.96
>28897.96
>[551] 28897.96 28897.96 25997.96 25997.96 25997.96 25997.96 24594.96
>24594.96 24594.96 24594.96 25965.00 25965.00 25965.00 30139.04 30139.04
>30139.04 26104.04 26104.04 26104.04 26104.04 26255.04 26255.04 26255.04
>26255.04 28887.00
>[576] 28887.00 28887.00 28887.00 28887.00 NA NA NA
>NA 25470.00 25470.00 25470.00 25470.00 20677.96 20677.96 20677.96 20677.96
>29227.00 29227.00 29227.00 29227.00 NA NA NA NA
>29543.96
>[601] 29543.96 29543.96 31080.00 31080.00 31080.00 27710.00 27710.00
>27710.00 27710.00 NA NA NA NA NA 29903.00
>29903.00 29903.00 29903.00 NA NA NA NA 24147.00
>24147.00 24147.00
>[626] 24147.00 23316.96 23316.96 23316.96 23316.96 27096.00 27096.00
>27096.00 27096.00 25543.00 25543.00 25543.00 25543.00 25543.00 NA
>NA NA NA 25131.04 25131.04 25131.04 29565.96 29565.96 29565.96
>29565.96
>[651] 28070.00 28070.00 28070.00 28070.00 28774.04 28774.04 28774.04
>28774.04 28073.00 28073.00 28130.00 28130.00 28130.00 28130.00 20038.00
>20038.00 20038.00 20038.00 27298.04 27298.04 27298.04 27298.04 27793.00
>27793.00 27793.00
>[676] 27793.00 25586.00 25586.00 25586.00 25586.00 26000.00 26000.00
>26000.00 26000.00 30577.04 30577.04 30577.04 30577.04 27194.04 27194.04
>27194.04 27194.04 23156.04 23156.04 23156.04 23156.04 23978.04 23978.04
>23978.04 23978.04
>[701] NA NA NA NA 24391.00 24391.00 24391.00
>24391.00 27152.96 27152.96 27152.96 27152.96 28852.00 28852.00 28852.00
>25419.00 25419.00 29212.00 29212.00 29212.00 29212.00 23660.00 23660.00
>23660.00 23660.00
>[726] 26022.96 26022.96 26022.96 26022.96 25566.00 25566.00 25566.00
>25566.00 25336.96 25336.96 25336.96 25336.96 26931.96 26931.96 26931.96
>26931.96 26758.00 26758.00 26758.00 26537.96 26537.96 26537.96 26537.96
>27026.00 27026.00
>[751] 27026.00 NA NA NA 24349.96 24349.96 24349.96
>24349.96 25960.00 25960.00 25960.00 25960.00 27276.00 27276.00 27276.00
>26826.96 26826.96 26826.96 26826.96 26428.96 26428.96 26428.96 26428.96
>26780.96 26780.96
>[776] 26780.96 26780.96 26301.00 26301.00 26301.00 26301.00 28385.96
>28385.96 28385.96 28385.96 27210.96 27210.96 27210.96 23704.00 23704.00
>23704.00 24160.04 24160.04 24160.04 24160.04 25703.96 25703.96 25703.96
>25703.96 25269.00
>[801] 25269.00 25269.00 25269.00 29886.96 29886.96 29886.96 NA
>NA NA NA NA NA
>attr(,"tzone")
>[1] ""
>
>Here are the values of sdate and dob that relate to the missing values in
>difftime(sdate,dob)
>
>> sdate[is.na(difftime(sdate,dob))]
> [1] "2002-02-28" "2002-02-28" "2002-02-28" "2002-02-28" "2002-07-30"
>"2002-07-30" "2002-07-30" "2002-07-30" "2002-07-30" "2003-06-17"
>"2003-06-17" "2003-06-17" "2003-10-30" "2003-10-30" "2003-10-30"
>"2002-07-22" "2002-07-22"
>[18] "2002-07-22" "2002-07-22" "2002-12-18" "2002-12-18" "2002-12-18"
>"2002-12-18" "2003-03-10" "2003-03-10" "2003-03-10" "2003-03-10"
>"2003-02-05" "2003-02-05" "2003-02-05" "2003-02-05" "2003-03-19"
>"2003-03-19" "2003-03-19"
>[35] "2003-03-19" "2003-05-29" "2003-05-29" "2003-05-29" "2003-05-29"
>"2003-08-13" "2003-08-13" "2003-08-13" "2003-08-13" "2003-11-03"
>"2003-11-03" "2003-11-03" "2003-11-03" "2003-11-03" "2002-06-25"
>"2002-06-25" "2002-06-25"
>[52] "2002-06-25" "2003-04-10" "2003-04-10" "2003-04-10" "2003-04-10"
>"2003-04-03" "2003-04-03" "2003-04-03" "2003-04-03" "2003-10-15"
>"2003-10-15" "2003-10-15" "2003-11-21" "2003-11-21" "2003-11-21"
>"2003-12-04" "2003-12-04"
>[69] "2003-12-04"
>> dob[is.na(difftime(sdate,dob))]
> [1] "1927-04-03" "1927-04-03" "1927-04-03" "1927-04-03" "1925-04-11"
>"1925-04-11" "1925-04-11" "1925-04-11" "1925-04-11" "1939-04-03"
>"1939-04-03" "1939-04-03" "1940-12-30" "1940-12-30" "1940-12-30"
>"1917-10-14" "1917-10-14"
>[18] "1917-10-14" "1917-10-14" "1925-04-16" "1925-04-16" "1925-04-16"
>"1925-04-16" "1927-04-05" "1927-04-05" "1927-04-05" "1927-04-05"
>"1939-04-08" "1939-04-08" "1939-04-08" "1939-04-08" "1938-10-24"
>"1938-10-24" "1938-10-24"
>[35] "1938-10-24" "1930-10-16" "1930-10-16" "1930-10-16" "1930-10-16"
>"1923-04-17" "1923-04-17" "1923-04-17" "1923-04-17" "1929-04-17"
>"1929-04-17" "1929-04-17" "1929-04-17" "1929-04-17" "1925-04-11"
>"1925-04-11" "1925-04-11"
>[52] "1925-04-11" "1931-04-02" "1931-04-02" "1931-04-02" "1931-04-02"
>"1929-04-18" "1929-04-18" "1929-04-18" "1929-04-18" "1917-10-22"
>"1917-10-22" "1917-10-22" "1928-03-28" "1928-03-28" "1928-03-28"
>"1928-04-09" "1928-04-09"
>[69] "1928-04-09"
>
>The values of dob here do not differ in any obvious way from those in the
>rest of the dob vector, where difftime(sdate,sob) gives sensible results.
>
>If I try to recompute the difftime, the result is the same.
>
>s1=sdate[is.na(difftime(sdate,dob))]
>d1=dob[is.na(difftime(sdate,dob))]
>difftime(s1,d1)
>Time differences in secs
> [1] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
>NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
>NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
>attr(,"tzone")
>[1] ""
>
>However, if I now create the first value of each missing vector manually,
>then difftime works:-
>
>js1=strptime('2002-02-28','%Y-%m-%d'); js1
>#[1] "2002-02-28"
>jb1=strptime('1927-04-03','%Y-%m-%d'); jb1
>#[1] "1927-04-03"
>difftime(js1,jb1)
>#Time difference of 27360 days
>
>So, it appears that strptime is handling these values differently in the
>vector, but manages them correctly one by one.
>
>I'm sorry if I'm being silly, but I can't see the problem. I'd be VERY
>grateful if someone could help me to find it and fix it.
>
>With many thanks in advance for your thoughts,
>
>Jonathan Williams
>
>______________________________________________
>R-help at r-project.org mailing list
>https://*stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide http://*www.*R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.
--
--------------------------------------
Don MacQueen
Environmental Protection Department
Lawrence Livermore National Laboratory
Livermore, CA, USA
925-423-1062
More information about the R-help
mailing list