"@ima -- thanks. The data differ with each other by 6 mths. The first was from oct-mar 2011 and other apr-sep 2010. I have done the things suggested by people here and now getting better rates."
AnalyticBridge
Data Intelligence, Business Analytics
Ayush Biyani's Groups
-
Analytic, Data Science a…
664 members
-
Indian Chapter
214 members
-
SAS Network
450 members
Ayush Biyani's Page
Latest Activity
Jozo Kovac commented on Ayush Biyani's blog post low accuracy in out of time dataset
"Ayush, remove predictors with diferent distributions between months & re-train model. Your performance will decrease, but prediction will be more stabile and more useful.
Another approach is to train your model on union of both time…"
Ralph Winters commented on Ayush Biyani's blog post low accuracy in out of time dataset
"Hari. I would consider using binary time series for a straight logistic regression problem. Alternatively you can try looking at Cox regression which uses a survival (hazard function) model instead of an logit model.
-Ralph Winters"
Miles Garnsey commented on Ayush Biyani's blog post low accuracy in out of time dataset
"I'm by no means an expert on this sort of thing, but surely defaulters' behaviour is going to change with the economic situation to some degree, was your training data from before the GFC for example, while your testing data was from…"
Hariharan Sunder commented on Ayush Biyani's blog post low accuracy in out of time dataset
"Ayush,
Does your modeling sample consist of data collected from only one period in time? If so the model may be necessarily hold good on a out of time sample. I think your modeling sample needs to be much more random so as to include effects of…"
Ralph Winters commented on Ayush Biyani's blog post low accuracy in out of time dataset
"From what you describe, time can be a factor and you need to look into a different modeling methodology. Especially if you are seeing some of the categorical variables change over time. Try looking into time series cross sectional…"
Ayush Biyani posted a blog post
low accuracy in out of time dataset
Hi guys..I am working on a logistic model. When I did out of sample validation, my percentage detection of the defaulters was 80%. The next I tried is out of time validation. To my dismay the accuracy(percent detection) came down to 33% this time. I am wondering and disappointed by what could have happened. I have profiled both the population and found differences in the distribution of few categorical variables. Please pour in your ideas as to what can be done to improve the accuracy in the…See More
Ayush Biyani replied to Christopher Bound's discussion CrowdANALYTIX Contest: Predicting Wine Quality
"Hi...what do we have to do for this contest ?? Is there a community that we first need to join for this contest. ??
Please tell us step by step."
Ayush Biyani replied to Vincent Granville's discussion I've added 7 new SAS jobs on the SAS Jobs Blog in the group SAS Network
"hi...I am interested in this..my linked in profile can be found easily by name "ayush biyani""
Ayush Biyani commented on Rachel's blog post How to start a statistical career for a graduate when everywhere is asking for years of experiences?
"Hi Rachel, I can fully understand your situation. Because I was in this situation some 1 yr back. You still have a Medical stats degree. I just had a BTech in Computer Science from a reputed Institute in India. I would just tell you what I did. I…"
Profile Information
- Short Bio:
- Deeply interested in analytics because of its link to maths and business..maintain a blog.
- My Website or LinkedIn Profile (URL):
- http://ayushb.blogspot.com
- Field of Expertise:
- Business Analytics, Predictive Modeling, Data Mining, Quant, Econometrics, Vizualization, Statistical Consulting
- Years of Experience in Analytical Role:
- 1
- Professional Status:
- Consultant
- Interests:
- Finding a New Position, Networking, New Venture, Other
- What is your Favorite Data Mining or Analytical Website?
- http://stats.stackexchange.com
- Your Company:
- Kie square consulting
- Industry:
- consulting
- How did you find out about AnalyticBridge?
- internet
Ayush Biyani's Blog
low accuracy in out of time dataset
Hi guys..I am working on a logistic model. When I did out of sample validation, my percentage detection of the defaulters was 80%. The next I tried is out of time validation. To my dismay the accuracy(percent detection) came down to 33% this time. I am wondering and disappointed by what could have happened. I have profiled both the population and found differences in the distribution of few categorical variables.
Please pour in your ideas as to what can be done to…
ContinuePosted on June 1, 2011 at 6:44am — 7 Comments
Comment Wall (1 comment)
- At 12:24am on May 25, 2011,
Divyabh Mishra said… That's odd. which browser are you using? Let me have your email id and we can add you in manually for now.
Divyabh
Top Content
Follow us
Big Data Jobs
© 2013 AnalyticBridge.com is a subsidiary and dedicated channel of Data Science Central LLC
