At a glance: Run an incrementality experiment to perform lift analysis of remarketing (retargeting) campaigns.
Related reading | Incrementality guide
Incrementality experiment checklist
Use this checklist to help set up your experiment.
| # | Description |
|---|---|
| 01 |
Define test objectives Formulate your hypothesis and decide what you want to measure. |
| 02 |
Split an audience into test and control groups In Audiences:
During an experiment, don't make any changes to the audience configuration. Changes to partners, audience name, or other settings will invalidate the statistical analysis. |
| 03 |
Launch the test In the Ad network:
The test group sent to the ad network is exposed to ads. The control group is not exposed to ads. |
| 04 |
Analyze the test results The test results show the incremental value of each group. |
| 05 |
Act on the test results Based on the findings,
|
Form a hypothesis
An incrementality experiment needs a hypothesis. Use these variables as a guide.
| Variables | Details and best practices |
|---|---|
| Hypothesis | Base it on the information in this table |
| Audience | Describe the audience |
| Audiences size |
Use an audience of at least 30,000 users and split them between 1-3 ad networks. Best practice: Find a large user base and a campaign with high conversion rates. A significant statistical base will contribute to the success of the experiment. |
| Campaign | Describe the campaign |
| Control group % | Best practice: 15 to 25% |
| Test group |
Best practice: Use 1-3 networks Test groups are allocated as a percentage. Decide which ad networks get the test groups. |
Example hypothesis
| Variables | Details |
|---|---|
| Hypothesis | The rate of test group users—who engage with the remarketing campaign and make a purchase—will be significantly higher than users in the control group. |
| Audience | Users who completed registration, but didn't make a purchase. |
| Audience size | 100,000 |
| Campaign | Campaign that provides a discount voucher to users who make a purchase during a specified period. |
| Control group % | 15% |
| Test group | Send test group to Network A and Network B. |
Add an audience and launch the experiment
It's recommended to let an experiment run for at least 3 weeks; this allows for ongoing user conversions such as in-app events and revenue.
1. Define the audience
Follow these steps and the incrementality tool will begin to record measurements.
- Go to Audiences, add an audience.
- In the Audiences connect tab:
- Connect the audience to 1-3 ad network partners.
- Enable Split audience.
- Set the size of the control group. Recommended: 15% or more.
2. Configure campaigns in ad networks
- On the day the audience is split, configure the campaign in the ad network.
- Note! Direct the campaign to target users uploaded by AppsFlyer Audiences; don't include users from any other source.
3. Use the Incrementality dashboard
Review results in the Incrementality dashboard (updated daily)
- The dashboard is updated daily.
- Results appear on the day after you split the audience.
- Raw-data incrementality reports are available via Data Locker.