[R] scan

[M. Edward (Ed) Borasky]

On Wed, 14 Mar 2001, Heberto Ghezzo wrote:

> Hello, I have a problem. I have to read a HUGE file which has to be
> line processed, so I would like to use scan like
> p <- scan(pfile,what=c(0,0,0,"",0,""),nlines=1)
> but it continues to read the first line, so I have to add a skip and
> increment the skip after each read.
> It takes forever to read a test file of 1 Mb, the real one is in 60's Mb
> Is there a way to read line by line an ASCII file without having to
> reread the part already read?
> Thanks for any help.

There are two fairly easy ways to do this sort of thing, depending on the type
of expertise available:

1. Load the huge file into a database with a good text file import capability,
like Microsoft Access, then use RODBC to read the data you're interested in
with a query. This is what I do for manual analysis; I have lots of huge files.

2. Write a Perl script to extract the relevant data to a smaller file and then
read the smaller file directly in R. This is what I do for my *automated*
analysis.

BTW, I've processed files of 120-odd MB this way on 400 MHz Pentium-class
machines with 128 MB of RAM.
--
znmeb at aracnet.com (M. Edward Borasky) http://www.aracnet.com/~znmeb

What is it about

 @replace(@string(@log(10000)*@atan(1),9),0,0, at char(112)&@char(105)&@char(61))

that you don't understand?

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._