Incrementality Testing

Attribution tells you which marketing activities touched the deals that closed. Incrementality tells you which marketing activities actually caused deals to close that would not have closed otherwise. The distinction is critical. A deal that would have closed without marketing intervention still shows marketing touchpoints in the attribution model. Marketing gets credit it did not earn. Incrementality testing isolates the causal impact by comparing outcomes between a group that received the marketing treatment and a control group that did not. The difference between the two is the incremental lift — the revenue that would not exist without the marketing activity.

1. Design the Holdout Randomly assign 10-15% of the target audience to a control group that receives no marketing treatment. The control group must be statistically representative. Random assignment is non-negotiable — any systematic bias invalidates the test.
2. Measure the Difference After the campaign period, compare conversion rates between the treatment and control groups. The difference is the incremental lift. A campaign with 12% conversion in the treatment group and 8% in the control produced 4% incremental lift — the other 8% would have happened anyway.
3. Calculate Incremental ROI Apply the incremental lift to the campaign cost. If the campaign cost $50K and produced $200K in revenue but only $80K was incremental, the incremental ROI is 60%, not 300%. This is the honest number.