At a glance: Incremental lift analysis measures the benefit derived from a remarketing (retargeting) campaign after taking into account organic cannibalization. To calculate lift, in AppsFlyer, you define an audience, select ad network partners to run the campaign, and then enable the ad network partner to receive the audience. AppsFlyer does the rest; we manage participants, test and control groups, partners, metrics, and more. Calculated results display in the Incrementality dashboard.
Incrementality analysis is used to make sure that remarketing campaigns have a positive contribution. It considers users who would have converted organically without engaging with a remarketing campaign. The AppsFlyer Incrementality solution has its foundations in the medical science intent-to-treat (ITT) experiment methodology.
Using ITT, researchers randomly split a given population between a control group who are not treated and a test group whom researchers intend to treat. It matters not if treatment is given or not. Researchers measure the efficacy of a given treatment by comparing the results of the groups.
Similarly, in remarketing, the lift metric measures campaign efficacy. It doesn't matter if a user in the target group engages with a campaign (receives treatment) because it is the intent that is relevant in calculating incrementality.
Statistical significance (p-value)
To make sure that the lift result isn't due to random events or chance, ITT methodology requires that the data be assessed for statistical significance using a p-value indicator.
The p-value indicates that experiment results are more than 90% likely to repeat themselves if you run a replicate of the experiment. A p-value of less than 0.1 (typically ≤ 0.1) is statistically significant. It indicates strong evidence against the null hypothesis—there is less than a 10% probability the null is correct (and the results are random). Therefore, the null hypothesis is rejected and the alternative hypothesis accepted.
To use the Incrementality dashboard:
- In AppsFlyer, go to Dashboard > Incrementality.
- Set filters to display the required metrics.
- Use display controls to:
- Select different metrics
- Display options according to those described in the following tables.
Audience selected. Can be used to target more than one app.
|First seen dates||
Date on which a user first matched audience rules and was then added to the audience.
|Targeted app||An app that benefits from the remarketing campaign efforts.|
|Media sources||Ad networks displaying ads to test group members.|
|Trend display chart/table form||
Change the chart/table display with these controls:
Incrementality raw data reports
Use Incrementality raw data reports to analyze the interaction of users with remarketing campaigns.
- Incrementality raw data reports contain row-level data of users included in Incrementality experiments. Download Incrementality raw data example files
- Report characteristics:
- Availability: Via Data Locker
- Data freshness: Data Locker Daily
Data Locker folder
Organic in-app events
In-app events non-organic
In-app events re-attributions
Uninstalls (This report is currently not populated)
- Users participating in the experiment—First-seen report
- In Audiences, rules are set characterizing users to include in the experiment.
- When a given user, is identified as matching the rules, the event is recorded in the first_seen report.
- Users are allocated randomly to a test or control group indicated by the is_control_group field.
- Users in the test group are allocated to a media source (pid_destination) for retargeting.
- User engagement within the app: User engagement with the app during the experiment is recorded in context-specific reports:
- Engagement type: Session or in-app event
- User attribution status when the user is first-seen: Organic, non-organic, re-attribution. For example, in the past, a user installed the app and was attributed to organic. As such, during an experiment, the attribution status is organic.
- Uninstalls: Users uninstalling the app during the experiment. Uninstall measurement must be active.
Data characteristics and fields
Field availability varies according to report type as indicated.
|Field||Description||First seen||In-app events||Sessions||Uninstalls|
|is_control_group||If true, the user is part of the control group||Y||Y||Y||Y|
|pid_destination||The media source the user is sent to||Y||Y||Y||Y|
|joined_audience_date||Date user first joined the audience||Y||Y||Y||Y|
|audience_name||Audience name (not unique)||Y||Y||Y||Y|
|tm||Hour of day||Y||Y||Y|
|timestamp||Event time stamp YYYY-MM-DD HH:MM||Y||Y|
|app_ids||App ids associated with the audience rules||Y|
|Field||Display name*||First seen||In-app events||Sessions||Uninstalls|
|advertising_id||Advertising ID (GAID)||Y||Y||Y||Y|
|customer_user_id||Customer user ID||Y||Y|
|event_revenue_currency||Event revenue currency||Y|
|event_revenue_u_s_d||Event revenue USD||Y|
|is_purchase_validated||Is receipt validated||Y|
|* According to raw data specification|
Traits and limitations
|Ad network access||Not available|
|Agency access||Not available|
|Agency transparency||Not applicable|
|Team member access||Yes, per account permissions.|