Variable Selection Methods in Regression: Many Statisticians Know Them, But Few Know They Produce Poorly Performing Models

Replies to This Discussion

Permalink Reply by Sharethram Hariharan (Shareth) on April 7, 2009 at 10:10am

In SAS enterprise Miner we typically use Decision Trees or Filter nodes to do variable selection and missing values imputation before we pass them on to Neural Nets or Regressions...Not sure how they do it statistically though. Would love to know.

▶ Reply to This

Permalink Reply by Sean Salleh on September 29, 2009 at 8:19pm

Aside from a statistical theory perspective, this comment applies well from an applications perspective in which the resulting regression model found from a variable selection method is difficult to interpret within the context of the problem. However, I imagine that other methods like CHAID, taking charge where usual regressions fail, would be an option as it may be more powerful in prediction and easier in interpretation.

▶ Reply to This

Permalink Reply by Bruce Ratner on September 30, 2009 at 2:07am

Sean:
I am not sure about other methods. From my research, I saw not even a hint of it!
It's just another thing to put on the 204 old list of OLS' cracks.
Bruce

▶ Reply to This

Permalink Reply by Sean Salleh on September 30, 2009 at 2:22am

Oh boy, I got scared the last time I saw that list ..more cracks than a 20-foot wall from the san francisco quake.

▶ Reply to This

Permalink Reply by Bruce Ratner on September 30, 2009 at 2:26am

Sean:
Good Show!
Bruce

▶ Reply to This