Incrementality for UA Guide

Incrementality for UA uses geo-based experimentation to measure true campaign impact in a scientifically rigorous way. Instead of segmenting users, we segment geographic areas into statistically similar exposed and holdout regions. These regions are selected based on size and historical app behavior. Ads run only in the exposed regions, while the holdout regions receive no exposure. By comparing the behavior of the two groups during the holdout period, we can measure the actual causal effect — not just correlation.

Geo-experiments follow a structured timeline:

• Pretest: Both groups are equally exposed to ads from the measured campaigns. During this time, we collect baseline data and build a model that captures the historical relationship between the exposed and holdout regions.
• Intervention (the experiment): Campaigns are activated in the exposed regions only. Holdout regions are prevented from viewing ads from the measured campaigns.
• Causal effect calculation: The difference between the observed outcomes in the holdout regions and the exposed regions during the experiment is summed over time and used to determine the cumulative impact.

Time-Based Regression (TBR)

At the heart of our geo-experimentation is Time-Based Regression (TBR). TBR is used to calculate the marketing lift by predicting what would have happened in the absence of a marketing intervention. This involves comparing the observed outcomes in an exposed region (where the intervention is made) with the predicted outcomes based on the historical relationship between a holdout region (not exposed to the intervention) and the exposed region.

How it works:

• Creating geo-designs of holdout and exposed regions:
  • Per app, AppsFlyer analyses historical app install or in-app event behaviour and uses TBR (explained in the next steps) on thousands of randomized sets of geographical holdout/exposed regions over multiple time periods to find the ones with the smallest difference.
• Pretest period:
  • During this time, before the intervention starts, data is collected for both the holdout and exposed regions.
  • A regression model is built to capture the relationship between the time series of the exposed region and the holdout region.
• Prediction of counterfactuals:
  • Using the model developed in the pretest period, the expected outcomes for the holdout region (if no intervention occurred) are predicted during the test period. These are called "counterfactuals".
• Test period:
  • For the campaigns that are part of the Incrementality experiment, the predetermined holdout regions are automatically excluded from viewing ads for the duration of the experiment.
• Causal effect calculation:
  • The causal effect of the intervention at each time point is the difference between the observed outcome in the holdout region and its counterfactual prediction.
• Cumulative effects:
  • These differences are summed up over time to determine the cumulative impact of the intervention (e.g., total increase in installs or conversions).
• Marketing lift:
  • The marketing lift is the cumulative effect that quantifies the incremental results driven by marketing activity compared to what would have been expected without it.

Incrementality for UA helps you solve key growth challenges by revealing the true business impact of your campaigns. Below are some common use cases where geo-based incrementality testing adds critical value.

1. Optimize budget allocation across ad platforms
   Challenge: Attribution metrics show similar performance across multiple platforms, but you’re unsure which is actually driving growth. Solution: Run parallel incrementality tests using multiple cells across platforms. Use the results to identify which platform causes real uplift and shift budget accordingly.
2. Validate performance of new channels or formats
   Challenge: You’re testing a new channel or ad platform, but early metrics are inconsistent or inflated due to targeting overlap. Solution: Use geo-testing to isolate the impact of the new channel from others in your mix. Determine whether it contributes net new installs or just cannibalizes existing ones.
3. Understand impact of branded campaigns
   Challenge: Branded campaigns perform well in attribution reports, but you’re not sure if users would have converted anyway. Solution: Geo-experiments can reveal whether branded search or display ads are actually influencing user behavior, or if they’re capturing existing demand.
4. Compare campaign strategies within the same platform
   Challenge: You want to test two creative strategies (e.g., video vs. carousel) or bidding types (e.g., maximize installs vs. ROAS) within the same network. Solution: Run a multi-cell experiment with each strategy assigned to a different cell. Compare the incremental lift of each and choose the winner.
5. Reallocate spend from saturated to scalable markets
   Challenge: Mature markets show diminishing returns and rising acquisition costs, but attribution data doesn’t clarify if you’re reaching new users. Solution: Geo-lift tests show whether campaigns in these markets are still incremental. If not, shift budget to untapped or undervalued regions with higher potential.

This section discusses the best campaigns to select for your experiments. Here are a few things to consider:

• Impact: Which test could influence your overall strategy the most? Often, that means starting with your largest initiatives since having a deeper understanding of how they affect growth will lead to meaningful optimizations.
• Conversion volume: Strong experiments need enough data. For each measured campaign, aim for a minimum of 5000 monthly app installs or events, according to the main KPI you would like to measure
• Exploration: Incrementality is an amazing tool when testing something new, such as entering a new market or launching a new channel.
• Having a clear hypothesis or test question: Like any other scientific experiment, spending some time on phrasing your main question will contribute to the effectiveness and actionability of your Incrementality experiment. For example, do you expect one channel to be more incremental than the other? How will it affect your marketing strategy if your hypothesis is proved or disproved?

Key experiment KPI outputs include:

• Lift (%): The causal impact of the measured marketing activity, measured as the additional conversions (e.g., installs or in-app events) your campaign actually drove, above and beyond what would have happened without it.
• Incremental conversions: The number of installs or events that are caused by the measured campaign.
• Cost per incremental install / Cost per incremental event: The average cost of each incremental conversion, based on the campaign’s ad spend.
• Incrementality factor: The ratio between incremental installs or events (as measured by the experiment) to the classic attribution installs or events (as measured in AppsFlyer's classic attribution).

See the full list of experiment KPIs in the dashboard section.

To make Incrementality insights actionable, AppsFlyer provides both the Lift % and campaign-level Incrementality KPIs for effective marketing decisions.

Optimizing based on Lift %:

A. No lift:
The ultimate goal of marketing is to generate incremental outcomes. If your campaign was measured to have no lift (holdout regions are performing at the same level as exposed regions), this means it is not adding value beyond what would have happened without it and requires further optimization in order to be meaningful for growth. If your campaign has no lift, consider the following:

• Shift budgets to more incremental campaigns or channels.
• Optimize: As with any other low-performing campaign, you can work to improve it by adjusting bids, goals, target audience, creatives, and other relevant factors. You can then run another Incrementality experiment following the update to monitor if incrementality has improved.
• Diagnose what factors might be causing low lift by checking tech setup, targeting, channel mix, placements, budgets, keywords, creatives, or campaign overlap. These learnings will be highly beneficial for optimizing towards incremental outcomes in the future.

B. Positive lift:
Positive lift means your campaign is generating incremental outcomes beyond what is achieved without it. As an initial step, you should validate that all your marketing efforts are generating positive lift. When lift is positive, consider the following:

• Increase investment in platforms/channels/campaigns to scale what’s proven to drive growth.
• Broaden reach by expanding audiences or locations to amplify impact.
• Identify growth factors to duplicate success and prioritize winning strategies.
• Compare campaigns or channels based on lift: Lift is a relative measure calculated by comparing holdout and exposed regions. As such, comparing campaigns based on lift is challenging. For example, if both campaigns have the same impact but one of them is reaching a significantly higher number of users or has a significantly higher amount of spend, its lift will be higher.
  When comparing, try and compare campaigns that are similar in spend and reach and are generally targeting the same geographical regions.
• If you are just starting out with Incrementality measurement for your app or business, you may not yet have a solid benchmark to determine the exact level of positive lift expected from a given campaign, channel, or investment. In this case, it is recommended to optimize using AppsFlyer’s campaign-level Incrementality KPIs, as described below.

Optimizing based on campaign-level incrementality KPIs:

A. Total incremental conversions and cost metrics:
In the same way it is used with classic attribution, the overall volume of incremental conversions and the ad spend invested to generate these conversions provide clear and actionable metrics for optimization. You can use the campaign-level incrementality conversions count and cost per incremental conversion to determine if the campaign meets your volume and ROI goals, and how it performs against other campaigns or channels.

B. Leveraging classic attribution and incrementality data side-by-side
For both the count of campaign-level conversions and cost per conversion, the Incrementality dashboard presents classic (last-touch) attribution data AND incrementality data side-by-side so you can learn the relationship between the two models and action your learnings at a larger scale.
When looked at separately, classic (“last-touch”) attribution and incrementality are two different attribution models, each one with its own advantages and disadvantages:

• Classic (“last-touch”) attribution is real-time, standardized, highly scalable, and granular. Therefore, it enables user-level optimization both from the advertiser end and through real-time ad platform algorithms. However, it is biased towards bottom-funnel engagement and does not provide scientific causal proof.
• Incrementality is a great tool to scientifically prove causal impact and is ideal for omnichannel measurement. However, it provides aggregated (campaign level) insights, and is periodic and harder to scale.

When used together, the two models provide a holistic view of marketing performance. AppsFlyer’s Incrementality Factor presents the relationship between classic attribution data that is the “baseline” used for ongoing marketing optimization, and incrementality data that is used to measure causal impact. Here is how you can use incrementality factors for optimization:

• Incrementality factor above 100%: Incrementality has measured more conversions than the conversions measured in classic attribution for the same campaign. For example, a 110% rate means incremental conversions are 10% higher than conversions measured using classic attribution. This increase in conversions also translates to a decrease in cost per conversion, since more conversions are generated for the same amount of spend.
  If the incrementality factor is above 100%, consider the following:
  • Re-evaluate budget allocation in accordance with the decrease in cost per incremental conversion.
  • Re-evaluate bids and optimization goals in accordance with cost per incremental conversion and incremental return on ad spend.
  • Implement the factor to similar campaigns ahead of making future marketing decisions. For example, if your video campaigns consistently show ~110% incrementality factor for app installs when measured in incrementality experiments, you can treat classic attribution app installs for similar campaigns as 10% higher when making marketing decisions (i.e., each 100 classic attribution installs are in fact closer to 110 installs).
• Incrementality factor below 100%: Incrementality has measured fewer conversions than the conversions measured in classic attribution for the same campaign. For example, a 90% rate means incremental conversions are 10% lower than conversions measured using classic attribution. This decrease in conversions also translates to an increase in cost per conversion, since fewer conversions are generated for the same amount of spend. If the incrementality factor is below 100%, consider the following:
  • Re-evaluate budget allocation in accordance with the increase in cost per incremental conversion.

•  
  • Re-evaluate bids and optimization goals in accordance with cost per incremental conversion and incremental return on ad spend.
  • Implement this factor in similar campaigns to inform future marketing decisions. For example, if your video campaigns consistently show a ~90% incrementality factor for app installs when measured in incrementality experiments, you can treat classic attribution app installs for similar campaigns as 10% lower when making marketing decisions (i.e., each 100 classic attribution installs are in fact closer to 90 installs).

Consider pausing experiments (before the default 30 days) if:

• Lift, Incrementality Factor, and Cost metrics have remained stable for over 7 days.
• There’s a business justification to obtain faster results at the expense of potentially less accurate or less “normalized” Incrementality metrics.

Consider extending experiments (beyond the default 30 days) if:

• Campaign volume was lower than expected and is anticipated to increase beyond the minimum requirement of daily incremental conversions.
• The conversion window for the measured event is exceptionally long, requiring additional time for users to convert.

The first step is creating the experiment.

To create the experiment:

1. Access Incrementality for UA by clicking Optimize > Incrementality for UA, in the AppsFlyer side menu. 

   

2. Click + New experiment.
    
3. Fill out the experiment form:
   1. Enter experiment name - Create a descriptive name for the experiment, then click Next.

   2. Add (or select) a geo design - Define the basis for your experiment by selecting the app, country, and main metric you want to test.
AppsFlyer will review your geo design for viability (the process takes about 24 hours). If approved, AppsFlyer will automatically allocate optimal exposed and holdout regions using geo-based methodology. Additionally, the geo design will be available for use in future experiments as well. Learn more

      

      Note: Please check back in 24 hours to see if your new geo design was approved. If not approved, it means it doesn’t currently meet the requirements for a valid experiment.
You can try submitting the design again at a later date because eligibility may change as your app gains more data.

      • Select app - Select the relevant app for the experiment.

      • Select country - Then, select the country where you want to run the experiment. Click Next.

        Note

        Each country has a certain number of cells available for your experiment. A cell consists of an exposed and holdout group pair that are structured to be similar to each other and to the entire country. You can use multiple cells in one experiment for a head-to-head comparison of different campaigns across media sources, or use them individually in separate experiments.

        The number of available cells per country depends on the country’s geographic size, population distribution, and app activity levels—factors that affect the ability to create statistically valid exposed/holdout pairs. Countries with more granular, active regions support more cells, enabling more experimentation.

      • Select the Main metric you want to measure. This can be installs or an in-app event.

       

   3. Set campaign details: 

• You'll need to select a Media source connection for each experiment cell. This is the media source where your campaign runs. If there are no connections, you can create a new connection.

• Then select the relevant campaigns to measure in the experiment.
  Incrementality experiments can be set to measure individual campaigns (for example, "How is the Google campaign performing vs. the Meta campaign") or the entire media source by selecting multiple campaigns within a single experiment cell (for example, "How is Google performing vs. Meta?").
  
  Choose between:
  Campaign level: to measure the impact of a single campaign within a media source
  Channel level: to measure the joint impact of multiple campaigns within a media source

  Note: If multiple campaigns are selected within one cell:

  • All selected campaigns will share the same holdout regions.

  • Incrementality insights will be presented at the cell level, and not at the individual campaign level.

    

• When selecting a campaign name, below you’ll see the minimum number of incremental installs or events (based on your selected main metric) required to measure your campaign’s positive impact with high statistical confidence. This is known as the “Minimum Detectable Effect” of the experiment. Make sure the selected campaign is expected to generate at least this number of conversions. 

  Tip: Choose a campaign that regularly meets or exceeds this minimum to ensure reliable, statistically valid results.

• You can add additional experiment cells if you want. To add: Click + Experiment cell and select the relevant connection and campaign. A cell consists of an exposed and holdout group pair that are structured to be similar to each other and to the entire country. You can use multiple cells in one experiment for a head-to-head comparison of different campaigns across media sources.
  
  After setting up the cells, click Next.

4. Review summary - Check that the experiment summary looks good, then click Launch experiment.
   The experiment begins. Initial results will display within 24 hours.

   Note

   Once the experiment is over, the connected campaigns will return to their previous geo-targeting settings. This ensures campaign continuity and consistency and lets you resume UA activities without needing to reconfigure targeting manually.

   Example
   Let’s say you ran an experiment in the United States using three designated exposed cities. During the experiment, your campaign was restricted to only those cities. When the experiment ends, the campaign will automatically resume its original targeting—such as “United States (all locations)”—as it was configured before the experiment began.

You can see the experiment results on the experiment dashboard.

Access the experiment dashboard

1. Go to your Incrementality for UA experiments by clicking Optimize > Incrementality for UA, in the AppsFlyer side menu. 
    

   

2. Click the specific experiment to access the experiment dashboard.

   

A tour of the dashboard

The top of the dashboard displays 2 visual graphs:

• The Lift: Exposed vs. holdout regions graph - Shows the incremental lift per experiment cell.
• The Experiment timeline: Exposed vs. holdout regions graph - Shows a side-by-side comparison of incremental installs per day of the experiment date range, per cell. 

The middle section of the dashboard displays the Experiment KPIs by campaign cell, media source, and campaign:

• Classic attribution installs - The number of installs the campaign contributed, as measured by classic attribution.
• Incrementality installs - The number of installs the campaign contributed, as measured by the experiment.
   
• Incrementality factor -The ratio between incremental installs (measured by the experiment) to classic attribution installs. 
For example, a 110% rate means incremental installs are 10% higher than installs measured using classic attribution.
• Classic attribution cost per install - The average cost of every attributed install, based on the campaign’s ad spend.
• Incrementality cost per install - The average cost of every incremental install, based on the campaign’s ad spend.
• Lift - How many more installs occurred because of your campaign, compared to what would have happened without it. Calculated as: Lift (%) = (actual performance in exposed group (excluded geos) 
- predicted performance in exposed group) / predicted performance in exposed group * 100 * -1.
• Confidence - The level of statistical confidence that the lift is greater than zero.
  
  For guidance on how to read the results, see Understanding experiment results and using the insights. 

The bottom of the dashboard is the Cumulative incremental installs graph which shows the cumulative incremental installs per cell.

Note: If Channel level measurement is selected (multiple campaigns within one cell):

• All selected campaigns will share the same holdout regions.
• Incrementality insights will be presented at the cell level, and not at the individual campaign level.

Your geo design is the basis for your experiments. You define it by selecting the app, country, and main metric you want to test in your experiments. Once you create a geo design, AppsFlyer will review it for viability. When approved, the geo design will be available for use in all your experiments. You can have up to 10 active geo designs at a given time. After creating geo designs, you can view and manage them.

To view and manage your geo designs:

In Incrementality for UA, click the Geo designs tab.

Your geo designs are organized by app and display the Country, Main metric, Status, Created by, Created on, and number of experiments based on the design.

Status

After creating your geo design, AppsFlyer will review it for viability (the process takes about 24 hours). If approved, AppsFlyer will automatically allocate optimal exposed and holdout regions using geo-based methodology. The geo design will be available for use in future experiments as well. Possible geo design statuses include:

• In review - Appsflyer is reviewing your geo design for eligibility. Check back in 24 hours to see if approved.
• Ready - The geo design is eligible and ready to use. You can start creating experiments.
• Unavailable - The geo design isn’t eligible based on the data we have for the current app, country, and measurement combination. Use another geo design. You can try resubmitting the design again (after 30 days). It may become eligible as more data becomes available.

Connections refer to the integration between Incrementality for UA and your media partners. They make it possible to link campaigns and run experiments with connected platforms.
 

To add media source connections:   

1. Access Incrementality for UA by clicking Optimize > Incrementality for UA, in the AppsFlyer side menu. 

2. Click the Connections tab, then + New connection.

   

3. Select a media source to connect to.

   

4. Enter your credentials to log in and create the connection.

You can view and manage all your experiments in one place.

To view and manage your experiments:

In Incrementality for UA, click the Experiments tab.

For each experiment, you can see the:

• Experiment name
• App  - sortable
• Country - sortable
• Media source
• Dates - The experiment date range
• Status - You can see the number of days left in the experiment or when it's completed.
• Lift

Status column

• Completed - Experiment period has ended
• Active - Experiment is currently running
• Pending - Experiment pending results (first 24 hours)
• In review - Design in review (up to 24 hours until answer)
• Ready - Design was reviewed and found eligible
• Unavailable - Design was reviewed and wasn't found eligible

Hover on the experiment to access theactions icon.

From here you can:

• View experiment details
• Extend experiment duration
• Stop experiment

To extend an experiment:

From the experiment, click theactions icon (on the upper-right) and select Extend experiment duration.

This will extend the experiment by an additional 14 days.

To stop an experiment:

From the experiment, click the ⠇actions icon (on the upper-right) and select Stop experiment.