Data Intelligence, Business Analytics
I would like to know your opinion about this subject, as lately a lot on emphasis has been put on this issue.
In telecom industry, CSPs are interested to determine if their marketing campaigns are successful, if they have actually achieved their goals. Analysts in this field support the idea that the use of a control group when analyzing and evaluating a campaign is needed in order to give reliability to your analysis results.
But how do you select a control group for a certain campaign?
What are the characteristics that this group should have?
Looking forward for your comments/opinions,
You should actually test the marketing campaign before you launch it by selecting test and control group samples. If there is a significant difference in the results for the test and control groups the marketing campaign should be successful when lauched to the entire target population.
I'm going to be fairly specific about this.
1) Set up the universe of the campaign, the selection criteria used to target customers
2) Identify the success criteria for the campaign
3) Make an educated guess as to the effect of the campaign; use this in power calculations to estimate the size of the control group.
4) It may be that you do not have enough customers to hold out an effective sample, or the needed sample may violate established policies. If so, make a decision about using a control group methodology.
5) Use a random number generator to identify x % of the campaign for the control group.
6) Make sure that the sample is representative.
6.1) If your campaign has multiple cells, make sure that the sample is pulled equally from all cells. You may want to pull the control group from each cell seperately.
6.2) Select a few critical dimensions, and then test the campaign group against the control group to make sure that there are no meaningful differences between the two.
The idea is to make sure that the campaign is the only difference between test and control, so resulting differences can be attirbuted to the campaign.
I have a similar doubt and I am new to this topic on how to do a post campaign Analysis.
Say I have some 100 retail store outlets which has one loyalty program for all these stores. The customers are segmented in 3 levels ('X', ‘Y’ and ‘Z’) . and I am launching a campaign for one of the (say 'ABC') retail store .
Let me know whether I am right in selecting the test and control group.
1) So now my intial aim is to form a test group who are ‘ABC’ retail customers who are active since last 1 year or so. Out of 3,00,000 (0.3 million customers) I had overall for all my 100 stores, I found around 50,000 customers were active for ‘ABC’ retail store since last 1 year. So I went ahead taking 50,000 as my target population.
2) Next my aim is to select the control group. For this I am blindly taking around 4000 customers as my control group from the 50,000 target population.
These leaves me around 46000 customers as test group population and 4000 as control group population.
1) Am I right in selecting test and control group population?
2) Is it compulsory that my test and control group should be of same proportion. I mean here do I need to split as 25000 each for test and control groups?
3) On what basis or variables do I need to select the control group? ( I have entire POS data of all my clients)
4) Do I need to check for normality test for control group? How to do?
5) How do I remove any outliers from the control group? How to remove?
6) Do I need to do some statistical analysis using t-test or Chi-square test for significance testing for both the test and control group, If needed on what parameter I need to check? Which test??
7) How do I calculate the incremental benefit from these two groups?
8) Finally what all the parameters I need to show in my Post-Campaign report? Right now I am showing total revenue, No of customers participated , revenue per customer.
What else I can show???
Please answer to these questions as early as possible. I am a newbie to this post-campaign analysis J
This will be of a great help to me!!!
So you've got your campaign target population selected (although 50k out of 3mm seems pretty narrow to me). The next step is: what measurable effect do you want the campaign to have on your customers? Once you've got that, that will tell you how to measure the campaign and also how big a population to have (although if you are starting off with 50k, I would not be surprised if the population so small that any test v control group will be noise).
I am not getting with What do you mean by "Measurable effect"?
I am looking at the metric total turnover generated by the campaign?
Please tell me how to calculate the incremental benefit? Here the 50K i mentioned is just sample target population (An example i gave), it can be of range from 50K to 150K as well.
If possible can you please answer my above posted questions ....thanks a lot for your time!!