Time series analysis and a control group may be used to evaluate the success of a direct marketing campaign whether it is delivered via Canada Post, or is an e-mail or search engine marketing campaign. A control group is a random sample of customers who do not receive a direct marketing offer and their behaviour may be compared to a similar, randomly generated group of customers who don’t receive one. This type of direct marketing campaign analysis is performed over time to compare two groups before, during and after a specific campaign period--whether days, weeks, months, or otherwise. Observing the behaviour of a control group versus a similar group of targeted customers allows one to calculate the incremental campaign revenue generated by, and the overall return on investment (ROI), or a direct marketing campaign.

As an example, let’s imagine that there is an e-mail database of 5,000 customers, 500 of whom (TARGET_GROUP) have been randomly targeted with a direct marketing offer. Each TARGET_GROUP customer has been offered a free pen for every time, within a one-week campaign period, that he or she buys a pad of paper via our website.

Before a direct marketing campaign analysis may be conducted, it is important to know the following: both our pen and pad of paper have long been on the market and the same offer is being promoted on TV during the e-mail campaign.

A time series, control group e-mail marketing campaign analysis will seek to answer the following questions:

1. How much more revenue was generated from pen and paper sales by the e-mail campaign?

2. Given the costs, what is the overall return on investment (ROI) for the e-mail marketing campaign?

TARGET_GROUP’s pen and paper buying behaviour may be compared to a randomly generated CONTROL_GROUP of 500 customers who receive no offer. Both groups are best evaluated on a week-to-week basis: before, during and after the campaign period. A $PRODUCT variable will capture the amount of pen and paper revenue that is generated from each TARGET_GROUP and CONTROL_GROUP customer.

Graphing the TARGET_GROUP and CONTROL_GROUP customers, over time, according to the $PRODUCT variable will provide an answer to question 1: how much revenue did the e-mail campaign generate. If, during the one week campaign period, TARGET_GROUP customers spent $100 more on pen and papers than CONTROL_GROUP customers, the e-mail campaign generated an incremental $100 in revenue.

Determining an overall ROI for the campaign requires that the costs involved to create and run it be subtracted from the $100 in revenue generated by it.

Let’s assume that the following costs were incurred in order to execute the e-mail marketing campaign:

Creating and testing the e-mail blast template - $5

Programming the website to accept online payments - $25

Staff time to plan and manage the e-mail campaign - $30

Since our total costs were $60, an analysis would reveal that the overall ROI for the e-mail campaign is $40.

This example provides a simple illustration of how a time series analysis using control groups may be used to evaluate the success of a direct marketing campaign.