[R] NA command in a 'for' loop

Richard O'Keefe r@oknz @end|ng |rom gm@||@com
Thu Apr 23 12:37:09 CEST 2020 

You now say that you "want to get rid of" the rows
where V13 is 0.
  d1 <- d[d$V13 != 0,]
returns you a new data frame d1 containing all the
rows of d where V13 is not 0.

On Tue, 21 Apr 2020 at 15:53, Helen Sawaya <helensawaya using hotmail.com> wrote:

> Thank you all for your input.
>
> This is an example of one data file (I have 74 data files):
>
> 2.90546E+11, threat,    1, 2, 1, 2, 1,        death,        stove,
>    NA,           NA,  205,    0,  394
> 2.90546E+11, threat,    2, 2, 2, 1, 1,    emaciated,    shortened,
>    NA,           NA,  205,    0,  502
> 2.90546E+11, threat,    3, 1, 1, 1, 2,     mutilate,     consider,
>    NA,           NA,  205,    1,  468
> 2.90546E+11, threat,    6, 1, 2, 2, 1,         weep,         shop,
>    NA,           NA,  203,    1,  345
> 2.90546E+11, threat,    9, 2, 1, 2, 2,    tormented,    easygoing,
>    NA,           NA,  205,    1,  373
> 2.90546E+11, threat,   10, 1, 2, 2, 2,        snake,        table,
>    NA,           NA,  205,    1,  343
> 2.90546E+11, threat,   11, 2, 2, 1, 1,       crisis,       faucet,
>    NA,           NA,  203,    1,  437
> 2.90546E+11, threat,   12, 1, 1, 1, 1,       victim,      utensil,
>    NA,           NA,  203,    1,  343
> 2.90546E+11, threat,   14, 1, 2, 2, 1,    depressed,    repentant,
>    NA,           NA,  203,    1,  441
> 2.90546E+11, threat,   15, 2, 2, 1, 2,         scum,         shoe,
>    NA,           NA,  205,    1,  475
>
> ​Column 13 has values of 0s and 1s which my cognitive task outputted.
> Column 14 is the reaction time (ms) data. I want to get rid of the rows
> that contain zeros so I thought I'd first replace zeros with NAs then use
> complete.cases function to get rid of the NAs. I also wanted to apply other
> functions so I included them all in a loop. All work fine except for the
> one where I try to turn the zeros to NAs.
>
> Jim when I tried your mockdata example, it worked fine. But when I
> translated it to my data, I still get zeros in the output. Can you identify
> any mistranslations I'm doing?
>
> txt.files<-list.files(".",pattern="dotprobe") #all my data files are text
> files in one folder
> for(tf in txt.files) {
>   d<-read.table(tf)
>   d[,13][d[,13]==0]<-NA #column 13 contains zeros
>   d<-d[ ,-c(10,11)] #get rid of columns 10 and 11
>   write.table(d,sub("[.]",".tlbs.",tf),quote=FALSE, row.names=FALSE)
> }
>
> That's an example of one of the output I get:
>
> V1 V2 V3 V4 V5 V6 V7 V8 V9 V12 V13 V14
> 2.90546E+11, threat, 1, 2, 1, 2, 1, death, stove, 205, 0, 394
> 2.90546E+11, threat, 2, 2, 2, 1, 1, emaciated, shortened, 205, 0, 502
> 2.90546E+11, threat, 3, 1, 1, 1, 2, mutilate, consider, 205, 1, 468
> 2.90546E+11, threat, 6, 1, 2, 2, 1, weep, shop, 203, 1, 345
> 2.90546E+11, threat, 9, 2, 1, 2, 2, tormented, easygoing, 205, 1, 373
> 2.90546E+11, threat, 10, 1, 2, 2, 2, snake, table, 205, 1, 343
>
> Columns 10 and 11 were deleted. But zeros were not replaced by NAs.
> After all the data cleaning, the functions I'm interested in including in
> the loop are: get_tlbs and summarize_bias (and these also work fine in my
> loop).
>
> Thanks again 🙂
> Sincerely
> Helen
> ________________________________
> From: Jim Lemon <drjimlemon using gmail.com>
> Sent: Tuesday, April 21, 2020 2:52 AM
> To: Rui Barradas <ruipbarradas using sapo.pt>
> Cc: Helen Sawaya <helensawaya using hotmail.com>; Michael Dewey <
> lists using dewey.myzen.co.uk>; r-help using R-project.org <r-help using r-project.org>
> Subject: Re: [R] NA command in a 'for' loop
>
> Hi Helen,
> Your problem may lie in using row.names=TRUE. I was puzzled when an
> extra column kept popping up in the output files. For reading in and
> replacing zeros with NAs, this seems to work:
>
> for(mockdata in 1:3) {
>  mdf<-data.frame(sample(2:20,10),sample(2:20,10),sample(0:1,10,TRUE))
>  write.table(mdf,file=paste0("threat",mockdata,".txt"),quote=FALSE,
>   row.names=FALSE,col.names=FALSE)
> }
> txt.files<-list.files(".",pattern="threat[1-3]")
> for(tf in txt.files) {
>  d<-read.table(tf)
>  d[,3][d[,3]==0]<-NA
>  write.table(d,sub("[.]",".tbls.",tf),quote=FALSE,row.names=FALSE)
> }
>
> Jim
>
> On Tue, Apr 21, 2020 at 7:57 AM Rui Barradas <ruipbarradas using sapo.pt> wrote:
> >
> > Hello,
> >
> > I believe the only way we have to see what is happening is for you to
> > post the output of
> >
> >
> > dput(head(d, 20))  # or 30
> >
> >
> > or, with d2 a subset of d that includes zeros,
> >
> >
> > dput(head(d2, 20))
> >
> >
> > Hope this helps,
> >
> > Rui Barradas
> >
> > Às 17:48 de 20/04/20, Helen Sawaya escreveu:
> > > I have one column that represents correct response versus error
> (correct
> > > is coded as 1 and error is coded as 0). Nowhere else in the dataset are
> > > there values of 0. The vector is treated as an integer.
> > >
> ------------------------------------------------------------------------
> > > *From:* Michael Dewey <lists using dewey.myzen.co.uk>
> > > *Sent:* Monday, April 20, 2020 7:35 PM
> > > *To:* Helen Sawaya <helensawaya using hotmail.com>; Rui Barradas
> > > <ruipbarradas using sapo.pt>; r-help using R-project.org <r-help using R-project.org>
> > > *Subject:* Re: [R] NA command in a 'for' loop
> > > Just a thought Helen but is x being treated as a real and what you
> think
> > > are zero and are printed as zero are in fact some very small number? If
> > > so you need to alter your test appropriately.
> > >
> > > Michael
> > >
> > > On 20/04/2020 17:25, Helen Sawaya wrote:
> > >> Thank you for your reply.
> > >>
> > >> I tried d[] <- lapply(d, function(x) {is.na(x) <- x == 0; x})
> > >> but I am still getting zeros instead of NAs in my output..
> > >>
> > >> I wonder if the problem is that some of my data files don't have any
> zeros (participants made no errors)..
> > >> ________________________________
> > >> From: Rui Barradas <ruipbarradas using sapo.pt>
> > >> Sent: Monday, April 20, 2020 9:05 AM
> > >> To: Helen Sawaya <helensawaya using hotmail.com>; r-help using R-project.org
> <r-help using R-project.org>
> > >> Subject: Re: [R] NA command in a 'for' loop
> > >>
> > >> Hello,
> > >>
> > >> Instead of
> > >>
> > >> d[d == 0] <- NA
> > >>
> > >> try
> > >>
> > >> d[] <- lapply(d, function(x) {is.na(x) <- x == 0; x})
> > >>
> > >>
> > >> Also, in the first for loop
> > >>
> > >> paste(i, sep = "")
> > >>
> > >> does nothing, it's the same as i.
> > >> And the same for
> > >>
> > >> (d2$V4 == 1) == TRUE
> > >>
> > >> Since (d2$V4 == 1)  already is FALSE/TRUE there is no need for
> > >>
> > >> (.) == TRUE
> > >>
> > >>
> > >> Hope this helps,
> > >>
> > >> Rui Barradas
> > >>
> > >>
> > >>
> > >> Às 20:52 de 19/04/20, Helen Sawaya escreveu:
> > >>> Dear R experts,
> > >>>
> > >>> I am using a 'for' loop to apply commands to multiple datasets (each
> file is one participant). The only one not working is the command that
> identifies zeros in my datasets and changes them to NAs. But when I look at
> the output, zeros ("0") are still present.  Surprisingly, the functions
> work fine when I apply them to a single
> > > dataset (outside the loop). I've tried:
> > >>>
> > >>> all.files <- list.files(".")
> > >>> txt.files <- grep("threat.txt",all.files,value=T)
> > >>>
> > >>> for(i in txt.files){
> > >>>     d <- read.table(paste(i,sep=""),header=F)
> > >>>     d[d==0] <- NA #replace zeros with NA
> > >>>     write.table(d, paste0(i,".tlbs.txt"), quote=FALSE,
> row.names=TRUE)}
> > >>>     d<-d[ ,-c(10,11)]
> > >>>     d2<-d[complete.cases(d), ]
> > >>>     d2$V4<-as.numeric(d2$V4)
> > >>>     congruent <- (d2$V4 == 1) == TRUE
> > >>>     x <- get_tlbs(d2$V14, congruent, prior_weights = NULL, method =
> "weighted", fill_gaps = FALSE)
> > >>>     write.table(x, paste0(i,".tlbs.txt"), quote=FALSE,
> row.names=TRUE)}
> > >>>
> > >>> I've also tried:
> > >>>
> > >>> for(i in txt.files){
> > >>>     d <- read.table(paste(i,sep=""),header=F)
> > >>>     if (0 %in% d)
> > >>>     {replace_with_na(d,replace = list(x = 0))} # replace zeros with
> NA
> > >>>     d<-d[ ,-c(10,11)]
> > >>>     d2<-d[complete.cases(d), ]
> > >>>     d2$V4<-as.numeric(d2$V4)
> > >>>     congruent <- (d2$V4 == 1) == TRUE
> > >>>     x <- get_tlbs(d2$V14, congruent, prior_weights = NULL, method =
> "weighted", fill_gaps = FALSE)
> > >>>     write.table(x, paste0(i,".summaryoutput.txt"), quote=FALSE,
> row.names=TRUE)}
> > >>>
> > >>> Thank you for your help.
> > >>> Sincerely
> > >>> Helen
> > >>>
> > >>>         [[alternative HTML version deleted]]
> > >>>
> > >>> ______________________________________________
> > >>> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > >>> https://stat.ethz.ch/mailman/listinfo/r-help
> > >>> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > >>> and provide commented, minimal, self-contained, reproducible code.
> > >>>
> > >>
> > >>        [[alternative HTML version deleted]]
> > >>
> > >> ______________________________________________
> > >> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > >> https://stat.ethz.ch/mailman/listinfo/r-help
> > >> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > >> and provide commented, minimal, self-contained, reproducible code.
> > >>
> > >>
> > >
> > > --
> > > Michael
> > > http://www.dewey.myzen.co.uk/home.html
> >
> > ______________________________________________
> > R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

	[[alternative HTML version deleted]]

More information about the R-help mailing list