[R] How to more efficently read in a big matrix

[jim holtman]

If you want to read only the alternate columns that contain numerics,
then you can probably use:

scan('yourfile', what=c(rep(list(NULL), list(0)), 243), flush=TRUE,
fill=TRUE, skip=1)

On Nov 10, 2007 12:25 AM, affy snp <affysnp at gmail.com> wrote:
> Hi Gabor,
>
> Thanks a lot!
>
> The header of the big file looks like as follows:
>
> probe_set
> WM_806_Signal_A
> WM_806_call
> WM_1716_Signal_A
> WM_1716_call
> ....
>
> I only need those columns with the header as like _Signal_A
>
> Can you suggest how to use sqldf?
>
> Thanks!
>
> Allen
>
>
> On Nov 9, 2007 11:47 PM, Gabor Grothendieck <ggrothendieck at gmail.com> wrote:
> > 1. You might be able to speed it up somewhat by specifying
> > colClasses=.
> >
> > 2. Another possibility is that the devel version of
> > the sqldf package provides an interface which simplifies reading a data file
> > into sqlite and from there into R.  This is particularly useful if you
> > don't want to read it all in.  See example 6 on the home page:
> > http://sqldf.googlecode.com
> >
> > 3. If it doesn't change and its ok to read it in slowly once then just
> > read it in slowly and save() it.  Then you can load()
> > it on subsequent runs which should be fast.
> >
> > On Nov 9, 2007 11:39 PM, affy snp <affysnp at gmail.com> wrote:
> >
> > > Dear list,
> > >
> > > I need to read in a big table with 487 columns and 238,305 rows (row names
> > > and column names are supplied). Is there a code to read in the table in
> > > a fast way? I tried the read.table() but it seems that it takes forever :(
> > >
> > > Thanks a lot!
> > >
> > > Best,
> > >    Allen
> > >
> >
> > > ______________________________________________
> > > R-help at r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > > and provide commented, minimal, self-contained, reproducible code.
> > >
> >
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



-- 
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem you are trying to solve?