[R] Remove highly correlated variables from a data frame or matrix
Ana Marija
@okov|c@@n@m@r|j@ @end|ng |rom gm@||@com
Thu Nov 14 23:25:59 CET 2019
HI Jim,
This:
colnames(calc.jim)[colSums(abs(calc.jim)>0.8)<3]
was the master take!
Thank you so much!!!
On Thu, Nov 14, 2019 at 3:39 PM Jim Lemon <drjimlemon using gmail.com> wrote:
>
> I thought you were going to trick us. What I think you are asking now
> is how to get the variable names in the columns that have at most one
> _absolute_ value greater than 0.8. OK:
>
> # I'm not going to try to recreate your correlation matrix
> calc.jim<-matrix(runif(100,min=-1,max=1),nrow=10)
> for(i in 1:10) calc.jim[i,i]<-1
> rownames(calc.jim)<-<-colnames(calc.jim)<-paste0("rs",1:10)
>
> Now that we have a plausible fake correlation matrix, all we have to
> do is extract the column names:
>
> colnames(calc.jim)[colSums(abs(calc.jim)>0.8)<2]
>
> Of course, what you really meant could have been, "I want the column
> names of the variables with at most one absolute value greater than
> 0.8 ignoring the diagonal values because I don't care about those". If
> so:
>
> colnames(calc.jim)[colSums(abs(calc.jim)>0.8)<3]
>
> Any more tricks?
>
> Jim
>
> On Fri, Nov 15, 2019 at 8:17 AM Ana Marija <sokovic.anamarija using gmail.com> wrote:
> >
> > what would be the approach to remove variable that has at least 2
> > correlation coefficients >0.8?
> > this is the whole output of the head()
> >
> > > head(calc.rho)
> > rs56192520 rs3764410 rs145984817 rs1807401 rs1807402 rs35350506
> > rs56192520 1.000 0.976 0.927 0.927 0.927 0.927
> > rs3764410 0.976 1.000 0.952 0.952 0.952 0.952
> > rs145984817 0.927 0.952 1.000 1.000 1.000 1.000
> > rs1807401 0.927 0.952 1.000 1.000 1.000 1.000
> > rs1807402 0.927 0.952 1.000 1.000 1.000 1.000
> > rs35350506 0.927 0.952 1.000 1.000 1.000 1.000
> > rs2089177 rs12325677 rs62064624 rs62064631 rs2349295 rs2174369
> > rs56192520 0.927 0.927 0.927 0.927 0.709 0.903
> > rs3764410 0.952 0.952 0.952 0.952 0.728 0.928
> > rs145984817 1.000 1.000 1.000 1.000 0.771 0.975
> > rs1807401 1.000 1.000 1.000 1.000 0.771 0.975
> > rs1807402 1.000 1.000 1.000 1.000 0.771 0.975
> > rs35350506 1.000 1.000 1.000 1.000 0.771 0.975
> > rs7218554 rs62064634 rs4360974 rs4527060 rs6502526 rs6502527
> > rs56192520 0.903 0.903 0.903 0.903 0.903 0.903
> > rs3764410 0.928 0.928 0.928 0.928 0.928 0.928
> > rs145984817 0.975 0.975 0.975 0.975 0.975 0.975
> > rs1807401 0.975 0.975 0.975 0.975 0.975 0.975
> > rs1807402 0.975 0.975 0.975 0.975 0.975 0.975
> > rs35350506 0.975 0.975 0.975 0.975 0.975 0.975
> > rs9900318 rs8069906 rs9908521 rs9908336 rs9908870 rs9895995
> > rs56192520 0.903 0.268 0.327 0.327 0.327 0.582
> > rs3764410 0.928 0.276 0.336 0.336 0.336 0.598
> > rs145984817 0.975 0.309 0.371 0.371 0.371 0.638
> > rs1807401 0.975 0.309 0.371 0.371 0.371 0.638
> > rs1807402 0.975 0.309 0.371 0.371 0.371 0.638
> > rs35350506 0.975 0.309 0.371 0.371 0.371 0.638
> > rs7211086 rs9905280 rs8073305 rs8072086 rs4312350 rs4313843
> > rs56192520 0.880 0.268 0.327 0.880 0.880 0.880
> > rs3764410 0.905 0.276 0.336 0.905 0.905 0.905
> > rs145984817 0.951 0.309 0.371 0.951 0.951 0.951
> > rs1807401 0.951 0.309 0.371 0.951 0.951 0.951
> > rs1807402 0.951 0.309 0.371 0.951 0.951 0.951
> > rs35350506 0.951 0.309 0.371 0.951 0.951 0.951
> > rs8069610 rs883504 rs8072394 rs4280293 rs4465638 rs12602378
> > rs56192520 0.582 0.903 0.582 0.582 0.811 0.302
> > rs3764410 0.598 0.928 0.598 0.598 0.836 0.311
> > rs145984817 0.638 0.975 0.638 0.638 0.879 0.344
> > rs1807401 0.638 0.975 0.638 0.638 0.879 0.344
> > rs1807402 0.638 0.975 0.638 0.638 0.879 0.344
> > rs35350506 0.638 0.975 0.638 0.638 0.879 0.344
> > rs9899059 rs6502530 rs4380085 rs6502532 rs4792798 rs4792799
> > rs56192520 0.302 0.309 0.834 0.251 0.063 0.063
> > rs3764410 0.311 0.318 0.858 0.259 0.080 0.080
> > rs145984817 0.344 0.352 0.902 0.291 0.086 0.086
> > rs1807401 0.344 0.352 0.902 0.291 0.086 0.086
> > rs1807402 0.344 0.352 0.902 0.291 0.086 0.086
> > rs35350506 0.344 0.352 0.902 0.291 0.086 0.086
> > rs4316813 rs148563931 rs74751226 rs8068857 rs8069441 rs77397878
> > rs56192520 0.006 0.006 0.006 0.006 0.006 0.006
> > rs3764410 0.006 0.006 0.006 0.006 0.006 0.006
> > rs145984817 0.006 0.006 0.006 0.006 0.006 0.006
> > rs1807401 0.006 0.006 0.006 0.006 0.006 0.006
> > rs1807402 0.006 0.006 0.006 0.006 0.006 0.006
> > rs35350506 0.006 0.006 0.006 0.006 0.006 0.006
> > rs75339756 rs4608391 rs79569548 rs4275914 rs11870422 rs8075751
> > rs56192520 0.006 0.006 0.006 0.044 0.007 0.004
> > rs3764410 0.006 0.006 0.006 0.042 0.005 0.005
> > rs145984817 0.006 0.006 0.006 0.047 0.002 0.015
> > rs1807401 0.006 0.006 0.006 0.047 0.002 0.015
> > rs1807402 0.006 0.006 0.006 0.047 0.002 0.015
> > rs35350506 0.006 0.006 0.006 0.047 0.002 0.015
> > rs11658904 rs138437542 rs80344434 rs7222311 rs7221842 rs7223686
> > rs56192520 0.003 0.004 0.004 0.033 0.009 0.000
> > rs3764410 0.004 0.004 0.004 0.031 0.007 0.000
> > rs145984817 0.010 0.004 0.004 0.035 0.011 0.005
> > rs1807401 0.010 0.004 0.004 0.035 0.011 0.005
> > rs1807402 0.010 0.004 0.004 0.035 0.011 0.005
> > rs35350506 0.010 0.004 0.004 0.035 0.011 0.005
> > rs78013597 rs74965036 rs78063986 rs118106233 rs117345712
> > rs56192520 0.004 0.004 0.004 0.004 0.005
> > rs3764410 0.004 0.004 0.004 0.004 0.006
> > rs145984817 0.004 0.004 0.004 0.004 0.005
> > rs1807401 0.004 0.004 0.004 0.004 0.005
> > rs1807402 0.004 0.004 0.004 0.004 0.005
> > rs35350506 0.004 0.004 0.004 0.004 0.005
> > rs113004656 rs9898995 rs4985718 rs9893911 rs79110942 rs7208929
> > rs56192520 0.004 0.033 0.033 0.023 0.004 0.023
> > rs3764410 0.004 0.031 0.031 0.021 0.004 0.021
> > rs145984817 0.004 0.035 0.035 0.025 0.004 0.025
> > rs1807401 0.004 0.035 0.035 0.025 0.004 0.025
> > rs1807402 0.004 0.035 0.035 0.025 0.004 0.025
> > rs35350506 0.004 0.035 0.035 0.025 0.004 0.025
> > rs12601453 rs4078062 rs75129280 rs76664572 rs78961289 rs146364798
> > rs56192520 0.004 0.001 0.004 0.004 0.004 0.004
> > rs3764410 0.004 0.002 0.004 0.004 0.004 0.004
> > rs145984817 0.004 0.001 0.004 0.004 0.004 0.004
> > rs1807401 0.004 0.001 0.004 0.004 0.004 0.004
> > rs1807402 0.004 0.001 0.004 0.004 0.004 0.004
> > rs35350506 0.004 0.001 0.004 0.004 0.004 0.004
> > rs76715413 rs4078534 rs79457460 rs74369938 rs76423171 rs74668400
> > rs56192520 0 0.004 0.004 0.002 0.004 0.004
> > rs3764410 0 0.004 0.004 0.001 0.004 0.004
> > rs145984817 0 0.004 0.004 0.005 0.004 0.004
> > rs1807401 0 0.004 0.004 0.005 0.004 0.004
> > rs1807402 0 0.004 0.004 0.005 0.004 0.004
> > rs35350506 0 0.004 0.004 0.005 0.004 0.004
> > rs75146120 rs1135237 rs9914671 rs117759512 rs4985696 rs16961340
> > rs56192520 0.004 0.003 0.009 0.004 0.009 0.004
> > rs3764410 0.004 0.003 0.007 0.004 0.007 0.004
> > rs145984817 0.004 0.003 0.011 0.004 0.011 0.004
> > rs1807401 0.004 0.003 0.011 0.004 0.011 0.004
> > rs1807402 0.004 0.003 0.011 0.004 0.011 0.004
> > rs35350506 0.004 0.003 0.011 0.004 0.011 0.004
> > rs17794159 rs4247118 rs78572469 rs12601193 rs2349646 rs2090018
> > rs56192520 0.001 0.033 0.002 0.004 0.020 0.033
> > rs3764410 0.002 0.031 0.001 0.004 0.019 0.031
> > rs145984817 0.001 0.035 0.005 0.004 0.022 0.035
> > rs1807401 0.001 0.035 0.005 0.004 0.022 0.035
> > rs1807402 0.001 0.035 0.005 0.004 0.022 0.035
> > rs35350506 0.001 0.035 0.005 0.004 0.022 0.035
> > rs12601424 rs4985701 rs8064550 rs2271521 rs2271520 rs11078374
> > rs56192520 0.004 0.033 0.033 0.004 0.033 0.014
> > rs3764410 0.004 0.031 0.031 0.004 0.031 0.012
> > rs145984817 0.004 0.035 0.035 0.004 0.035 0.016
> > rs1807401 0.004 0.035 0.035 0.004 0.035 0.016
> > rs1807402 0.004 0.035 0.035 0.004 0.035 0.016
> > rs35350506 0.004 0.035 0.035 0.004 0.035 0.016
> > rs4985702 rs1124961 rs11652674 rs3924340 rs112450164 rs7208973
> > rs56192520 0.033 0.003 0.002 0.001 0.004 0.033
> > rs3764410 0.031 0.003 0.001 0.002 0.004 0.031
> > rs145984817 0.035 0.003 0.005 0.001 0.004 0.035
> > rs1807401 0.035 0.003 0.005 0.001 0.004 0.035
> > rs1807402 0.035 0.003 0.005 0.001 0.004 0.035
> > rs35350506 0.035 0.003 0.005 0.001 0.004 0.035
> > rs9910857 rs78574480 rs8072184 rs12602196 rs6502563 rs3744135
> > rs56192520 0.006 0.004 0.014 0.004 0.033 0.004
> > rs3764410 0.005 0.004 0.012 0.004 0.031 0.004
> > rs145984817 0.002 0.004 0.016 0.004 0.035 0.004
> > rs1807401 0.002 0.004 0.016 0.004 0.035 0.004
> > rs1807402 0.002 0.004 0.016 0.004 0.035 0.004
> > rs35350506 0.002 0.004 0.016 0.004 0.035 0.004
> > rs148779543 rs77689691 rs41319048 rs117340532 rs78647096 rs77712968
> > rs56192520 0 0.004 0.004 0.002 0.004 0.004
> > rs3764410 0 0.004 0.004 0.001 0.004 0.004
> > rs145984817 0 0.004 0.004 0.005 0.004 0.004
> > rs1807401 0 0.004 0.004 0.005 0.004 0.004
> > rs1807402 0 0.004 0.004 0.005 0.004 0.004
> > rs35350506 0 0.004 0.004 0.005 0.004 0.004
> > rs16961396 rs80054920 rs7206981 rs4985740 rs3803762 rs77103270
> > rs56192520 0.004 0.004 0.033 0.023 0.004 0.002
> > rs3764410 0.004 0.004 0.031 0.021 0.004 0.001
> > rs145984817 0.004 0.004 0.035 0.025 0.004 0.005
> > rs1807401 0.004 0.004 0.035 0.025 0.004 0.005
> > rs1807402 0.004 0.004 0.035 0.025 0.004 0.005
> > rs35350506 0.004 0.004 0.035 0.025 0.004 0.005
> > rs7207485 rs77342773 rs3826304 rs3744126 rs7210879 rs7211576
> > rs56192520 0.029 0.004 0.004 0.004 0.023 0.006
> > rs3764410 0.027 0.004 0.004 0.004 0.021 0.005
> > rs145984817 0.031 0.004 0.004 0.004 0.025 0.002
> > rs1807401 0.031 0.004 0.004 0.004 0.025 0.002
> > rs1807402 0.031 0.004 0.004 0.004 0.025 0.002
> > rs35350506 0.031 0.004 0.004 0.004 0.025 0.002
> > rs117967362 rs75978745 rs6502564 rs9894565 rs36079048 rs8076621
> > rs56192520 0.004 0.004 0.007 0.017 0 0.004
> > rs3764410 0.004 0.004 0.005 0.015 0 0.004
> > rs145984817 0.004 0.004 0.009 0.019 0 0.004
> > rs1807401 0.004 0.004 0.009 0.019 0 0.004
> > rs1807402 0.004 0.004 0.009 0.019 0 0.004
> > rs35350506 0.004 0.004 0.009 0.019 0 0.004
> > rs7218795 rs3803761 rs12602675 rs7208065 rs4985705 rs8080386
> > rs56192520 0.026 0.032 0 0.018 0.014 0.003
> > rs3764410 0.024 0.029 0 0.015 0.011 0.002
> > rs145984817 0.028 0.034 0 0.021 0.016 0.004
> > rs1807401 0.028 0.034 0 0.021 0.016 0.004
> > rs1807402 0.028 0.034 0 0.021 0.016 0.004
> > rs35350506 0.028 0.034 0 0.021 0.016 0.004
> > rs8065832 rs2018781 rs1736221 rs1736220 rs1736217 rs1708620
> > rs56192520 0.008 0.039 0.003 0.003 0.021 0.009
> > rs3764410 0.006 0.037 0.002 0.002 0.019 0.007
> > rs145984817 0.010 0.042 0.004 0.004 0.024 0.011
> > rs1807401 0.010 0.042 0.004 0.004 0.024 0.011
> > rs1807402 0.010 0.042 0.004 0.004 0.024 0.011
> > rs35350506 0.010 0.042 0.004 0.004 0.024 0.011
> > rs1708619 rs1736216 rs76319098 rs1736215 rs1736214 rs1708617
> > rs56192520 0.009 0.024 0.017 0.012 0.019 0.029
> > rs3764410 0.007 0.021 0.016 0.009 0.016 0.026
> > rs145984817 0.011 0.026 0.018 0.014 0.022 0.031
> > rs1807401 0.011 0.026 0.018 0.014 0.022 0.031
> > rs1807402 0.011 0.026 0.018 0.014 0.022 0.031
> > rs35350506 0.011 0.026 0.018 0.014 0.022 0.031
> > rs12602831 rs12602871 rs1736213 rs1736212 rs76045368 rs34518797
> > rs56192520 0.000 0.000 0.015 0.029 0.001 0.001
> > rs3764410 0.001 0.001 0.013 0.026 0.001 0.001
> > rs145984817 0.000 0.000 0.018 0.031 0.000 0.000
> > rs1807401 0.000 0.000 0.018 0.031 0.000 0.000
> > rs1807402 0.000 0.000 0.018 0.031 0.000 0.000
> > rs35350506 0.000 0.000 0.018 0.031 0.000 0.000
> > rs11078378 rs8079562 rs8065774 rs8066090 rs41337846 rs1736209
> > rs56192520 0.043 0.001 0.001 0.029 0.000 0.029
> > rs3764410 0.041 0.001 0.001 0.026 0.001 0.026
> > rs145984817 0.046 0.000 0.000 0.031 0.000 0.031
> > rs1807401 0.046 0.000 0.000 0.031 0.000 0.031
> > rs1807402 0.046 0.000 0.000 0.031 0.000 0.031
> > rs35350506 0.046 0.000 0.000 0.031 0.000 0.031
> > rs1736208 rs12949822 rs76246042 rs12600635 rs55689224 rs1736207
> > rs56192520 0.043 0.043 0.000 0.000 0.000 0.015
> > rs3764410 0.041 0.041 0.001 0.001 0.001 0.013
> > rs145984817 0.046 0.046 0.000 0.000 0.000 0.018
> > rs1807401 0.046 0.046 0.000 0.000 0.000 0.018
> > rs1807402 0.046 0.046 0.000 0.000 0.000 0.018
> > rs35350506 0.046 0.046 0.000 0.000 0.000 0.018
> > rs1708626 rs1736206 rs9896078 rs16961474 rs1708627 rs1736205
> > rs56192520 0.015 0.015 0.001 0.001 0.017 0.021
> > rs3764410 0.013 0.013 0.001 0.001 0.014 0.019
> > rs145984817 0.018 0.018 0.000 0.001 0.020 0.024
> > rs1807401 0.018 0.018 0.000 0.001 0.020 0.024
> > rs1807402 0.018 0.018 0.000 0.001 0.020 0.024
> > rs35350506 0.018 0.018 0.000 0.001 0.020 0.024
> > rs1708628 rs7220577 rs2294155 rs1736204 rs1736203 rs1736202
> > rs56192520 0.011 0.011 0.000 0.021 0.014 0.014
> > rs3764410 0.009 0.009 0.000 0.019 0.011 0.011
> > rs145984817 0.013 0.013 0.001 0.024 0.016 0.016
> > rs1807401 0.013 0.013 0.001 0.024 0.016 0.016
> > rs1807402 0.013 0.013 0.001 0.024 0.016 0.016
> > rs35350506 0.013 0.013 0.001 0.024 0.016 0.016
> > rs12937908 rs1736200 rs1708623 rs1708624 rs9894884 rs9901894
> > rs56192520 0.009 0.009 0.008 0.007 0.009 0.009
> > rs3764410 0.007 0.007 0.006 0.005 0.007 0.007
> > rs145984817 0.011 0.011 0.010 0.008 0.011 0.011
> > rs1807401 0.011 0.011 0.010 0.008 0.011 0.011
> > rs1807402 0.011 0.011 0.010 0.008 0.011 0.011
> > rs35350506 0.011 0.011 0.010 0.008 0.011 0.011
> > rs9903294 rs2472689 rs1630656 rs111478970 rs3182911 rs7219012
> > rs56192520 0.008 0.011 0.007 0.007 0.008 0.000
> > rs3764410 0.006 0.009 0.005 0.005 0.006 0.000
> > rs145984817 0.010 0.013 0.008 0.008 0.010 0.001
> > rs1807401 0.010 0.013 0.008 0.008 0.010 0.001
> > rs1807402 0.010 0.013 0.008 0.008 0.010 0.001
> > rs35350506 0.010 0.013 0.008 0.008 0.010 0.001
> > rs9890657 rs12453455 rs12947291 rs150267386 rs16961493 rs11652745
> > rs56192520 0.009 0.007 0.008 0.013 0.000 0.009
> > rs3764410 0.007 0.005 0.006 0.012 0.000 0.007
> > rs145984817 0.011 0.008 0.010 0.014 0.001 0.011
> > rs1807401 0.011 0.008 0.010 0.014 0.001 0.011
> > rs1807402 0.011 0.008 0.010 0.014 0.001 0.011
> > rs35350506 0.011 0.008 0.010 0.014 0.001 0.011
> > rs9907107 rs8070574 rs4985759 rs3866959 rs7219248 rs6502568
> > rs56192520 0.009 0.009 0.009 0.011 0.009 0.011
> > rs3764410 0.007 0.007 0.007 0.009 0.007 0.009
> > rs145984817 0.011 0.011 0.011 0.013 0.011 0.013
> > rs1807401 0.011 0.011 0.011 0.013 0.011 0.013
> > rs1807402 0.011 0.011 0.011 0.013 0.011 0.013
> > rs35350506 0.011 0.011 0.011 0.013 0.011 0.013
> > rs7220275 rs12450037 rs7225876 rs9892352 rs4985760 rs6502569
> > rs56192520 0.009 0.008 0.007 0.011 0.011 0.011
> > rs3764410 0.007 0.006 0.005 0.009 0.009 0.009
> > rs145984817 0.011 0.010 0.008 0.013 0.013 0.013
> > rs1807401 0.011 0.010 0.008 0.013 0.013 0.013
> > rs1807402 0.011 0.010 0.008 0.013 0.013 0.013
> > rs35350506 0.011 0.010 0.008 0.013 0.013 0.013
> > rs1029830 rs2012954 rs1029832 rs2270180 rs8072402 rs7221553
> > rs56192520 0.009 0.011 0.008 0.000 0.009 0.011
> > rs3764410 0.007 0.009 0.006 0.000 0.007 0.009
> > rs145984817 0.011 0.013 0.010 0.001 0.011 0.013
> > rs1807401 0.011 0.013 0.010 0.001 0.011 0.013
> > rs1807402 0.011 0.013 0.010 0.001 0.011 0.013
> > rs35350506 0.011 0.013 0.010 0.001 0.011 0.013
> > rs145597919 rs150772017 rs2041393 rs6502578 rs11078382 rs9912109
> > rs56192520 0.013 0.013 0.005 0.005 0 0.005
> > rs3764410 0.012 0.012 0.004 0.004 0 0.004
> > rs145984817 0.014 0.014 0.006 0.006 0 0.006
> > rs1807401 0.014 0.014 0.006 0.006 0 0.006
> > rs1807402 0.014 0.014 0.006 0.006 0 0.006
> > rs35350506 0.014 0.014 0.006 0.006 0 0.006
> > rs12601631 rs11869054 rs11869079 rs9912599 rs7220057 rs9896970
> > rs56192520 0 0 0 0 0 0
> > rs3764410 0 0 0 0 0 0
> > rs145984817 0 0 0 0 0 0
> > rs1807401 0 0 0 0 0 0
> > rs1807402 0 0 0 0 0 0
> > rs35350506 0 0 0 0 0 0
> > rs34121330 rs34668117 rs67773570 rs242252 rs955893 rs28583584
> > rs56192520 0.000 0.000 0 0.002 0.001 0.013
> > rs3764410 0.001 0.001 0 0.003 0.001 0.005
> > rs145984817 0.000 0.000 0 0.002 0.001 0.004
> > rs1807401 0.000 0.000 0 0.002 0.001 0.004
> > rs1807402 0.000 0.000 0 0.002 0.001 0.004
> > rs35350506 0.000 0.000 0 0.002 0.001 0.004
> > rs9944423 rs7217764 rs11651957 rs73978990 rs8071007 rs56044345
> > rs56192520 0.013 0.011 0.011 0.011 0.011 0.011
> > rs3764410 0.005 0.004 0.004 0.004 0.004 0.004
> > rs145984817 0.004 0.003 0.003 0.003 0.003 0.003
> > rs1807401 0.004 0.003 0.003 0.003 0.003 0.003
> > rs1807402 0.004 0.003 0.003 0.003 0.003 0.003
> > rs35350506 0.004 0.003 0.003 0.003 0.003 0.003
> > rs17804843
> > rs56192520 0
> > rs3764410 0
> > rs145984817 0
> > rs1807401 0
> > rs1807402 0
> > rs35350506 0
> >
> >
> > On Thu, Nov 14, 2019 at 2:59 PM Abby Spurdle <spurdle.a using gmail.com> wrote:
> > >
> > > That's assuming your data was returned by head().
> >
> > ______________________________________________
> > R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
More information about the R-help
mailing list