[R] reading big arrays from C

[Roger Bivand]

On Thu, 29 Mar 2001, Heberto Ghezzo wrote:

> hello.
> I am trying to read a big file with different sections each with a 
> different format, actually it is a map in format *.e00 of MapView.
> I can read the whole file with no problems from R using scan()
> but it takes too long, some files are 50 meg and have to be read line 
> by line to catch the section codes.
> So if I do the reading from C would it be faster?
> if it is, how do I manage with the vectors of values to be returned, 
> now I just rbind(old, new) at each point because I do not know 
> before hand how many points are there going to be.
> What will you advise me to try? read 500 points say return to R 
> rbind to previous , zero the array and call C again to keep on 
> reading? or I can do something like rbind from C?
> Thanks for any help.
> .

Some flexible solutions to your question are available using the new
file(), readLines(), and - for binary - readBin() functions in base. In
addition, you may find the Rstreams package convenient. Perhaps
unfortunately, *.e00 (Arc/Info export format) is quite complicated, and
could (maybe) best be pre-processed using Awk or Perl, which would chop
the sections up into separate files - which would be easier to read into
R. Please let me know how you get on - reading *.e00 and *.shp files is
also important for trying to get a map() function working.

Roger

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Breiviksveien 40, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 93 93
e-mail: Roger.Bivand at nhh.no
and: Department of Geography and Regional Development, University of
Gdansk, al. Mar. J. Pilsudskiego 46, PL-81 378 Gdynia, Poland.

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._