[R] ff problems

Bond, Stephen Stephen.Bond at cibc.com
Tue Apr 3 19:55:47 CEST 2012


this works:
bigcl <- read.table(file="bigCL1.csv",sep=',',header=T , colClasses=c(rep("factor",3),"numeric","NULL","integer",
                        "numeric","integer","NULL","numeric","NULL"),nrow=1000)

this doesn't:
bigcl <- read.table.ffdf(file="bigCL1.csv",sep=',',header=T ,
                          colClasses=c(TERMGROUP="factor",SN="factor",
                            INS="factor",INCENTIVE="numeric",SRC="character",FLAG="integer",
                            RELAGE="numeric",BURNSUM="integer",prod="character",BAL="numeric",origbal="NULL")
                          )
Error in ff(initdata = initdata, length = length, levels = levels, ordered = ordered,  :
  vmode 'character' not implemented

neither this:
> bigcl <- read.table.ffdf(file="bigCL1.csv",sep=',',header=T  ,
+                          colClasses=c(TERMGROUP="factor",SN="factor",
+                            INS="factor",INCENTIVE="numeric",SRC="NULL",FLAG="integer",
+                            RELAGE="numeric",BURNSUM="integer",prod="NULL",BAL="numeric",origbal="NULL")
+                          )
Error in repnam(colClasses, colnames(x), default = NA) :
  the following argument names do not match'SRC','prod','origbal'


if I load as is without classes then:
> bigcl <- read.table.ffdf(file="bigCL1.csv",sep=',',header=T)
> bigcl$term <- factor(bigcl$term)
Error in sort.list(y) : 'x' must be atomic for 'sort.list'
Have you called 'sort' on a list?

attempting to save it:
> ffsave(bigcl,file="fcl")
Error in system(cmd, input = filelist, intern = TRUE) : 'zip' not found

attempting to get rid of the unwanted columns:
> bigcl <- bigcl[,c(-5,-9,-11)]
Error: cannot allocate vector of size 129.8 Mb
In addition: Warning messages:
1: In class(df) <- "data.frame" :
  Reached total allocation of 1535Mb: see help(memory.size)
2: In class(df) <- "data.frame" :
  Reached total allocation of 1535Mb: see help(memory.size)
3: In class(df) <- "data.frame" :
  Reached total allocation of 1535Mb: see help(memory.size)
4: In class(df) <- "data.frame" :
  Reached total allocation of 1535Mb: see help(memory.size)

 ff was supposed to be gentle on RAM usage??

I am attaching the first 1000 lines from the csv file.
thanks a lot.


Stephen



More information about the R-help mailing list