[R] step, leaps, lasso, LSE or what?

[Frank, Murray]

Hi,

I am trying to understand the alternative methods that are available for
selecting
variables in a regression without simply imposing my own bias (having "good
judgement"). The methods implimented in leaps and step and stepAIC seem to 
fall into the general class of stepwise procedures. But these are commonly 
condemmed for inducing overfitting.

In Hastie, Tibshirani and Friedman "The Elements of Statistical Learning"
chapter 3, 
they describe a number of procedures that seem better. The use of
cross-validation 
in the training stage presumably helps guard against overfitting. They seem 
particularly favorable to shrinkage through ridge regressions, and to the
"lasso". This
may not be too surprising, given the authorship. Is the lasso "generally
accepted" as 
being a pretty good approach? Has it proved its worth on a variety of
problems? Or is 
it at the "interesting idea" stage? What, if anything, would be widely
accepted as 
being sensible -- apart from having "good judgement".

In econometrics there is a school (the "LSE methodology") which argues for
what
amounts to stepwise regressions combined with repeated tests of the
properties of 
the error terms. (It is actually a bit more complex than that.) This has
been coded in 
the program PCGets:
(http://www.pcgive.com/pcgets/index.html?content=/pcgets/main.html) 
If anyone knows how this compares in terms of effectiveness to the methods
discussed in 
Hastie et al., I would really be very interested. 

Cheers,
Murray

Murray Z. Frank
B.I. Ghert Family Foundation Professor
Strategy & Business Economics
Faculty of Commerce
University of British Columbia
Vancouver, B.C.
Canada V6T 1Z2

phone: 604-822-8480
fax: 604-822-8477
e-mail: Murray.Frank at commerce.ubc.ca  

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request at stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._