Social Network For Analytic Professionals
-
Sharethram Hariharan (Shareth)
- Male
- Stillwater, OK
- United States
- Share
- Blog Posts
- Discussions (4)
- Groups (3)
- Photos
- Photo Albums
- Videos
Sharethram Hariharan (Shareth)'s Groups
-
Analytical Techniques
185 members
-
Data Mining
186 members
-
Optimization
46 members
Sharethram Hariharan (Shareth)'s Discussions
Missing values in target variable
Replied Apr. 25, 2009
Missing values in target variable
Replied Apr. 13, 2009
Missing values in target variable
Replied Apr. 11, 2009
Sharethram Hariharan (Shareth)'s Page
Latest Activity
Hello,
I am not sure if this is the right place to ask this question but I am currently working on Supply chain Disruption Management. Specifically, I would like to model the impact of very low frequency but high impact events analytically - Not in…
don't quite understand why you need to do this at least not for the dependant variable. Obviously, for the independant variables, we need to account for missing values so as not to reduce our dataset to something that is irrelevant for the exercise.…
yes i need to include records where the dependant variable are missing.
My question to you is whether or not you want to include records in your analytical file where the target or dependant variable values are missing??
The data on which we need to build a model itself has about 6500 missing values in the target variable.There are only 120 non missing values in the target variable.
Can we do something like this??
Use a model (Say Model A) to predict the the missing…
To answer the question, you need ton know the actual physical mechanism that lead to the data being missing. Since you are changing the data, you need to know what the data means in the first place. Let me give a few examples.
-- The data is a medi…
in this case the structural missing values mean the target is not supposed to have values for the particular row. If event A happens then only event B (target) occurs which is similar to conditional probability .Based on the combination of input var…
Good point, Bill. It could easily be censored observations.
Any solution depends on why the target is missing in the first place.
Okay, this sounds like a classic case of censored observations. There is a regression technique called a Tobit model that is specifically geared towards this type of problem. SAS and a few other packages incorporate the ability to model this type of…
What do you mean by 'weird predicted values'?
If you're got 120 non-missing targets and are scoring 5500 cases, then even if the distributions are identical the larger sample will have cases that are much farther from the mean that the smaller samp…
Think about what you are doing for a second.
You are talking about building a model on the 120 cases, scoring the other 90% of the cases, and then (I presume) building a model on the full data set. What you'll get back is the original model with ab…
Can you explain a little about why the values are missing? There are different procedures that can be used to handle this if the values are missing at random, but if they are not missing at random then that complicates the situation.
Also, I'd stro…
hi,
I would need a little bit of information before trying to help- what do you mean by "structurally missing"- That phrase is very important as some people use it incorrectly. The following link has some guidelines (but please note that they are n…
Hi Arup,
I guess I would atleast have to do the data analysis part and use the "modeling" itself to be a very insignificant perecentage of my report. We already know that data prep is most important in data mining but I dont have any other choice r…
hi Shareth, im wondering if over 90% of the data for the target variable is missing, should we go forward with this analysis?
Profile Information
- Short Bio:
- Ph.d student in Industrial engineering and Management, focus is on supply chain risk modeling and mitigation., areas of interest: Operations Management including the interface of marketing and operations, inventory management, Data Mining/Predictive Analytics, Applied Optimization, Simulation and Game Theory....would like to work in the industry as a consultant and probably move to academia and consult part-time later on...
- Field of Expertise:
- Operations Research, SAS, Other
- Years of Experience in Analytical Role:
- 1
- Professional Status:
- Student
- Interests:
- Finding a New Position, Networking, New Venture, Other
- Your Company:
- Oklahoma State University
- Industry:
- Supply Chain and Operations Management
- How did you find out about AnalyticBridge?
Comment Wall (1 comment)
- At 10:53am on April 7, 2009,
Yi-Chun Tsai said…
- Hi, Sharethram:
It is nice to be friend with you. So, you are a PhD student in IE. How far have you been in your PhD study now? You seem to have used SAS Enterprise Miner before but I am wondering where do you get access from? When I was a student or even I am working full time now, I still don't have access to SAS EM.
Featured
Advertisement
Analytic Jobs
Most Visited
© 2010 Created by Vincent Granville
