[R] check for new files in a given directory

[(Ted Harding)]

On 28-Jul-09 13:40:31, Barry Rowlingson wrote:
> On Tue, Jul 28, 2009 at 12:36 PM, Ted
> Harding<Ted.Harding at manchester.ac.uk> wrote:
>> However, this got me looking into '?list.files, and I see there
>> (R version 2.9.0 (2009-04-17)):
>>
>> _recursive: logical. Should the listing recurse into directories?
>>
>> But:
>>
>> _Directories are included only if 'recursive = FALSE'.
>>
>> Surely the latter is the wrong way round, and should be
>>
>> _Directories are included only if 'recursive = TRUE'.
> 
>  by 'included' it means 'returned'. If you do 'recursive=TRUE' it
> scans recursively for files and only files. If you do
> "recursive=FALSE" it returns files and directories in the specified
> directory.
> 
> Makes it tricky to figure out a complete directory tree since empty
> directories won't appear at all if recursive=TRUE. You'd have to
> implement your own recursive search based on
> list.files(d,recursive=FALSE) and then testing for directoriness....
> 
>  Sucky, unless there's a better way...
> Barry

Thanks for the clarification, Barry! I hadn't appreciated all that
(I think the wording of ?list.files is misleading here).

I agree that it can leave you falling between two stools in the
sort of situation you describe. Which, as it happens, leads me back
to the use of system("...") in Unix/Linux systems. For example, to
list all the directories (without their files) in the current
directory you could do:

  system("find . -type d -print")

which finds (recursively) everything in and under the current
directory (".") which is of type directory ("d").

Likewise, the 'ls' command with suitable options (perhaps combined
with judicious 'grep'ping) can enable you to select what you want.
For example, if something is planting data files with extension
".dat" from time to time,

  system("ls -tr *.dat")

would list all files with extension ".dat" in time ("-t") order
reversed ("r") (most recent last). Again, "find" could be useful.
Suppose the last one of such files you accessed was lastfile.dat;
then:

  system("find -maxdepth 0 -newer lastfile.dat -name '*.dat' -print")

would find, in the current directory only ("-depth 0") all files
with extension ".dat" (-name '*.dat'") which are newer than the
given file "lastfile.dat".

It is for reasons of flexibility like this that I use system() for
this kind of thing anyway (with 'intern=TRUE' if the results are to
be saved in a variable), so I hadn't looked into list.files() before!

Ted.

--------------------------------------------------------------------
E-Mail: (Ted Harding) <Ted.Harding at manchester.ac.uk>
Fax-to-email: +44 (0)870 094 0861
Date: 28-Jul-09                                       Time: 15:24:09
------------------------------ XFMail ------------------------------