Replies to This Discussion

Permalink Reply by Tom Wolfer on September 27, 2010 at 2:51pm

So, you did or did not eliminate independent variables that highly correlate with each other? If you did not, then, try that. How much time did you put into shaping your data: outliers, binning, sampling etc?

• ▶ Reply

Permalink Reply by Tom Wolfer on September 27, 2010 at 2:59pm

Am I also correct to assume that your Pearson Chi-Square Goodness of Fit is low? When you look at your Odds Ratio table, have you still included variables in the model that are irrelevant?

• ▶ Reply

Permalink Reply by Hariharan Sunder on September 27, 2010 at 11:06pm

Hi Tom,

I used stepwise regression procedure to eliminate independent variables but most of my independent variables are significant (p<0.001) so I eliminated only a few insignificant variables.
Binning: I have binned data based on preliminary Univariate Analysis. I have binned only the demographics data and kept continuous variable as it is. Should i try changing my binning?

Also i have Age values missing for almost 30% of my data so i created a separate group called Unspecified , is this correct way to do?

• ▶ Reply

Permalink Reply by Purnendu Maji on September 28, 2010 at 12:36am

Hariharan,
Before using stepwise regression to eliminate the independent variable you should eliminate the multicolinearity effect. You know multicolinearity produce over-estimate and p-value going to be least. To get better performance, please remove multicolinearity first. In that case you can proceed with standardization the data.
If possible you can remove serial correlation as well.

Please do some emphasize on HL statistic also.

• ▶ Reply

Permalink Reply by Hariharan Sunder on September 28, 2010 at 3:26am

What would be a good method to detect multi-collinearity and correaltion among variables. I have lots of categorical variables and a few continuos variables.
I can use Proc Reg VIF option but need to recode the categorical to dummy variables.
Can anybody suggest a better way to detect correlation and multi-collimearity?

P.S. I have access to only Base SAS

• ▶ Reply

Permalink Reply by Tom Wolfer on September 28, 2010 at 7:43am

Tell me about your continuous variables. Are there outliers? Do they approach a normal distribution? It may help if you apply some function - like a LN or LOG or EXP - to your continuous variables to make them more normally distributed. If I am correct, regressions like variables to be as close to normally distributed as possible. As far as the Age variable is concerned, I think that I would see if I can use another variable in the datafile to 'estimate' the missing values. If this does not work, then, I would exclude the Age variable from modeling altogether - may just be too many missing values.

• ▶ Reply

Permalink Reply by MANISH NEGI on September 28, 2010 at 12:00pm

Hi
I recommend that you shlould read the following web page on Logistice regression assumptions which is though for SPSS but will be certainly useful to you:-
http://faculty.chass.ncsu.edu/garson/PA765/logistic.htm

• ▶ Reply

Permalink Reply by Ralph Winters on September 28, 2010 at 6:42pm

Also check your residuals to see if they are random. Your model may be missing something.

-Ralph Winters

• ▶ Reply

Permalink Reply by paul d on October 12, 2010 at 12:59am

Hi, when evaluating predictions, look at the initial breakdown in the data, because while you can get a good overall hit rate (i use 80% as a simple rule of thumb), looking at the data, what was your sensitivity and specificity. In other words, does your model classify both sets of conditions (outcome a and outcome b) you are modelling well? Having a high percentage in one group, and getting them classified correctly can really make your overall hit rate misleading.

I would chek your residuals (the difference between your expected as a probability) and the observed, and see which cases you are misclassifying, and which ones you are misclassifying really badly,and perhaps then try and profile them.

Also, remember that statistical significance can be boosted by sample size (power), and if you have a lot of cases, your predictors can be significanct. I would check the odds ratios etc, to see how much the presence of each predictor changes the overall odds of falling into one group or another

The only advantage i see to using model fitting stats like this is to compare different models (:

PS - don't use backward or forward elimination, or stepwise procedures, try to compile a rationale for including and or excluding certain variables, i would only use this as a last resort, also see if you can find some freeware to run tree algorithms on your data to see the impact of different models (logistic is not the be all and end all of modelling), rattlle in R is a nice GUI you could look at, and it removes the programming learning curve, and offers logisitc, boosting, random forests, svm etc which enables comparisons and classification rates

• ▶ Reply

Permalink Reply by Jane Hu on July 28, 2011 at 2:47pm

I have the same issues with this stat. I think it is sample size issue. 

 

I used VIF option under proc reg to make sure those variables entered into logistic model do not highly correlated. So correlation is not an issue.

 

My logistic model also has very high KS value.  

 

I suspect it is binning issue when dealing with very small group of responders. Sometime, low respond % also tends to generates logstics model with high KS. H-L test is like Chi-square test performed on 10 group of bins.

 

1 responder falling into a different bin can change a lot outcome on this chi-square test if responders are very thin. 

• ▶ Reply

Permalink Reply by Nur Andi on March 8, 2012 at 8:55pm

I'm also confused about this test for binary logistic and have similar problem with Hariharan. Thank you for advices. Significant H-L stat on my model has been solved follow suggestion on this treads.

• ▶ Reply