At a glance: Incremental lift analysis measures the benefit derived from a remarketing (retargeting) campaign after taking into account organic cannibalization. To calculate lift, in AppsFlyer, you define an audience, select ad network partners to run the campaign, and then enable the ad network partner to receive the audience. AppsFlyer does the rest; we manage participants, test and control groups, partners, metrics, and more. Calculated results display in the Incrementality dashboard.
Related reading | Running an incrementality experiment | Audiences
Incrementality
Incrementality analysis is used to make sure that remarketing campaigns have a positive contribution. It considers users who would have converted organically without engaging with a remarketing campaign. The AppsFlyer Incrementality solution has its foundations in the medical science intent-to-treat (ITT) experiment methodology.
Using ITT, researchers randomly split a given population between a control group who are not treated and a test group whom researchers intend to treat. It matters not if treatment is given or not. Researchers measure the efficacy of a given treatment by comparing the results of the groups.
Similarly, in remarketing, the lift metric measures campaign efficacy. It doesn't matter if a user in the target group engages with a campaign (receives treatment) because it is the intent that is relevant in calculating incrementality.
Calculating lift
Statistical significance (p-value)
To make sure that the lift result isn't due to random events or chance, ITT methodology requires that the data be assessed for statistical significance using a p-value indicator.
The p-value indicates that experiment results are more than 90% likely to repeat themselves if you run a replicate of the experiment. A p-value of less than 0.1 (typically ≤ 0.1) is statistically significant. It indicates strong evidence against the null hypothesis—there is less than a 10% probability the null is correct (and the results are random). Therefore, the null hypothesis is rejected and the alternative hypothesis accepted.
Guide to running an experiment
Incrementality dashboard
To use the Incrementality dashboard:
- In AppsFlyer, go to Dashboard > Incrementality.
- Set filters to display the required metrics.
- Use display controls to:
- Select different metrics
- Display options according to those described in the following tables.
| Component | Description |
|---|---|
|
|
|
| Audience |
Audience selected. Can be used to target more than one app. |
| First seen dates |
Date on which a user first matched audience rules and was then added to the audience. |
| Targeted app | An app that benefits from the remarketing campaign efforts. |
| Media sources | Ad networks displaying ads to test group members. |
| Component | Description | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Selected metric |
|
||||||||
| Group size |
|
||||||||
| Significance |
Significance:
|
| Component | Description |
|---|---|
| Trend display chart/table form |
Change the chart/table display with these controls:
|
Incrementality raw data reports
Use Incrementality raw data reports to analyze the interaction of users with remarketing campaigns.
- Incrementality raw data reports contain row-level data of users included in Incrementality experiments. Download Incrementality raw data example files
- Report characteristics:
- Availability: Via Data Locker
- Data freshness: Data Locker Daily
| Category | Report name |
Data Locker folder |
|---|---|---|
|
First-seen |
First-seen users |
incrementality_first_seen_users |
|
In-app events |
Organic in-app events |
incrementality_organic_inapps |
|
In-app events non-organic |
incrementality_inapps | |
|
In-app events re-attributions |
incrementality_inapps_reattr | |
|
Sessions
|
Organic sessions |
incrementality_organic_sessions |
|
Sessions non-organic |
incrementality_sessions | |
|
Sessions re-attributions |
incrementality_sessions_reattr | |
|
Uninstalls |
Uninstalls (This report is currently not populated) |
incrementality_uninstalls |
Report logic
- Users participating in the experiment—First-seen report
- In Audiences, rules are set characterizing users to include in the experiment.
- When a given user, is identified as matching the rules, the event is recorded in the first_seen report.
- Users are allocated randomly to a test or control group indicated by the is_control_group field.
- Users in the test group are allocated to a media source (pid_destination) for retargeting.
- User engagement within the app: User engagement with the app during the experiment is recorded in context-specific reports:
- Engagement type: Session or in-app event
- User attribution status when the user is first-seen: Organic, non-organic, re-attribution. For example, in the past, a user installed the app and was attributed to organic. As such, during an experiment, the attribution status is organic.
- Uninstalls: Users uninstalling the app during the experiment. Uninstall measurement must be active.
Data characteristics and fields
Field availability varies according to report type as indicated.
| Field | Description | First seen | In-app events | Sessions | Uninstalls |
|---|---|---|---|---|---|
| is_control_group | If true, the user is part of the control group | Y | Y | Y | Y |
| pid_destination | The media source the user is sent to | Y | Y | Y | Y |
| audience_id | Unique identifier | Y | Y | Y | Y |
| joined_audience_date | Date user first joined the audience | Y | Y | Y | Y |
| audience_name | Audience name (not unique) | Y | Y | Y | Y |
| tm | Hour of day | Y | Y | Y | |
| timestamp | Event time stamp YYYY-MM-DD HH:MM | Y | Y | ||
| app_ids | App ids associated with the audience rules | Y |
| Field | Display name* | First seen | In-app events | Sessions | Uninstalls |
|---|---|---|---|---|---|
| advertising_id | Advertising ID (GAID) | Y | Y | Y | Y |
| android_id | Android ID | Y | Y | ||
| app_id | App ID | Y | Y | Y | Y |
| app_name | App name | Y | Y | ||
| app_version | App version | Y | Y | ||
| appsflyer_id | Appsflyer ID | Y | Y | Y | |
| revenue_alt | App-specific currency | Y | |||
| bundle_id | Bundle ID | Y | Y | ||
| country | Country code | Y | Y | Y | |
| currency | Currency code | Y | Y | Y | |
| customer_user_id | Customer user ID | Y | Y | ||
| brand | Device brand | Y | Y | ||
| device_category | Device category | Y | Y | ||
| model | Device model | Y | Y | ||
| device_model | Device model | Y | Y | ||
| device_type | Device type | Y | Y | ||
| event_name | Event name | Y | Y | Y | |
| event_revenue | Event revenue | Y | |||
| event_revenue_currency | Event revenue currency | Y | |||
| event_revenue_u_s_d | Event revenue USD | Y | |||
| event_time | Event time | Y | Y | Y | |
| event_value | Event value | Y | Y | ||
| idfa | IDFA | Y | Y | Y | Y |
| idfv | IDFV | Y | Y | ||
| imei | IMEI | Y | Y | ||
| is_purchase_validated | Is receipt validated | Y | |||
| os_version | OS version | Y | Y | ||
| platform | Platform | Y | Y | Y | |
| sdk_version | SDK version | Y | Y | ||
| * According to raw data specification | |||||
Traits and limitations
| Trait | Remarks |
|---|---|
| Ad network access | Not available |
| Agency access | Not available |
| Agency transparency | Not applicable |
| Time zone | UTC |
| Currency | USD |
| Organic data | Yes |
| Non-organic data | Yes |
| Data freshness |
Daily |
| Historical data |
N/A |
| Team member access | Yes, per account permissions. |