Cost ETL—granular campaign cost in a bucket

Data written to the cloud service or bucket has the directory and file structure described:

• When Cost ETL completes writing to a directory, a flag is set by creating a success file. This is always the most recent timestamp in the directory. 
• The number of folders/files is as follows:
  • Summary: Up to 4 version folders per day.
    • Each version folder contains parquet files with 7 days of data.
  • Geo or Channel report: Each report contains up to 4 version folders per day.
    • Each version folder contains parquet files containing data with numbering starting from 1.
  • [Closed beta] All cost: Based on the geo dimension, contains up to 4 files per day.

Data Locker connection directory structure

<client-bucket>/<DL-generated-partition>/<connection-name>/t=<report-name>/dt=<YYYY-MM-DD>/v=<1/2/3/4>/<parquet file number>

Cost ETL connection directory structure

<advertiser bucket name>/cost_etl/version/dt=<yyyy-mm-dd>/b=<n>/<dimension>/<file name/<parquet file number>

Example: For the first data pull of June 23, 2020, the directory and file name structure is as follows: /bucket-name/cost_etl/v1/dt=2020-06-23/b=1/geo/part-00001

Fields/Dimensions

Metrics

This configuration procedure must be performed by an admin user.

Before you start:

Cost ETL setup requires:

• A cloud service set up with permission for AppsFlyer to write data to it. This means you need to set up either 1 or more of the following cloud services:
  • AWS bucket
  • GCS bucket
  • [Beta] Azure Blob
  • [Beta] Yandex bucket
  • BigQuery data warehouse
  • Snowflake warehouse
• AppsFlyer admin privileges and access to the AppsFlyer UI to complete Cost ETL setup.

To set up Cost ETL: 

1. In AppsFlyer, from the side menu, go to Export > Cost ETL.
2. Select or set up a Data connection:
   • If you already have data connections set up, in the Connection dropdown, select the ones you want for Cost ETL.
      
   • If you don't have a data connection yet, click + New connection in Data Locker and create a connection there:
      
     1. Name your connection.
         
     2. Select a cloud service data destination. Do one of the following:
        • Select S3.
           
          1. Enter your AWS S3 bucket name. af- prefix is mandatory, and should be entered manually.
          2. Click Test connection.
          3. Verify that an error message indicating that the bucket path is invalid isn't displayed.
          4. Select whether to Make this connection compatible with Adobe Experience Platform.
          5. Click Save.
          6.  In the Cost ETL setup, in the Connection dropdown, select the cloud service you created.
        • Select GCS.
           
          1. Enter your GCS bucket name.
          2. Click Test connection.
          3. Verify that an error message indicating that the bucket path is invalid isn't displayed.
          4. Select whether to Make this connection compatible with Adobe Experience Platform.
          5. Click Save.
          6.  In the Cost ETL setup, in the Connection dropdown, select the cloud service you created.
        • [Beta] Select Azure.
           
          1. Enter your Connection name, Storage account name, and Key.
          2. Verify that an error message indicating that the bucket path is invalid isn't displayed.
          3. Select whether to Make this connection compatible with Adobe Experience Platform.
          4. Click Save.
          5.  In the Cost ETL setup, in the Connection dropdown, select the cloud service you created.
        • [Beta] Select Yandex.
           
          1. Enter your Bucket name, Access key, and Secret key.
          2. Verify that an error message indicating that the bucket path is invalid isn't displayed.
          3. Select whether to Make this connection compatible with Adobe Experience Platform.
          4. Click Save.
          5.  In the Cost ETL setup, in the Connection dropdown, select the cloud service you created.
        • Select BigQuery.
           
          1. Enter your BigQuery project ID and dataset name.
          2. Click Test connection.
          3. Verify that an error message indicating that the bucket path is invalid isn't displayed.
          4. Click Save.
          5.  In the Cost ETL setup, in the Connection dropdown, select the cloud service you created.
        • Select Snowflake.
           
          1. Enter your Snowflake region and account ID.
          2. Click Test connection.
          3. Verify that an error message indicating that the bucket path is invalid isn't displayed.
          4. Click Save.
          5.  In the Cost ETL setup, in the Connection dropdown, select the cloud service you created.
3. Click Next.
    
4. Set the report Version schedule (UTC):
    
   1. Select how many reports to receive every day (1, 2, 3, or 4). 
   2. Select what time to receive the reports. Learn more
   3. Click Next.
5. Set up the report details: 
   1. Select 1 or more apps to include in reports. Select Add all future apps to automatically include apps you add in the future.
       
   2. Select at least 2 reports to receive:
       
      • Summary: Generalized report that provides high-level data for easier and faster consumption. Enabled by default.
      • Geo: Data broken down by geo for all media sources contained in that report. For example data for Google Ads is further broken down by geo: USA, Canada, France, etc.
      • Channel: Data broken down by channel for all media sources contained in that report. For example for Google Ads, data is broken down further by YouTube, Shopping, SERP, and Gmail.
      • [Closed beta] All cost report: Based on the geo dimension. Includes cost data for all marketing activity on all platforms, including for apps/platforms not added in AppsFlyer (in these cases, the app ID is marked as unknown.
   3. Select dimensions to be included in the reports. Note: If you chose to receive the Geo report, the geo dimension is included by default. If you chose to receive the Channel report, the channel dimension is included by default. 
       
   4. Select the metrics to be included in the reports. 
       
6. Click Apply.

You can edit or change connections to get data sent to different (or additional) places. 

Note: After switching to an existing connection, you will not be able to revert to your previous Cost ETL connection.

Edit connection

To edit connection details:

1. In AppsFlyer, from the side menu, go to Export > Cost ETL.
2. In the Data connection section, either:
   • For Data Locker connections, in the Connection dropdown, hover over the connection you want to edit. The connection details display. Click Edit in Data Locker.
      
   • For Cost ETL connections, click the Connection details dropdown. The connection details display.
      
3. Follow the on-screen instructions to edit your connection details.
4. Click Save changes or Apply as prompted.

Change Data Locker connection

To change from one Data Locker connection to another:

1. In AppsFlyer, from the side menu, go to Export > Cost ETL.
2. In the Data connection section, either:
   • If you already have the Data Locker connections you want set up, in the Connection dropdown, select the ones you want for Cost ETL.
      
   • If you don't have the Data Locker connection you want set up yet, click + New connection in Data Locker and create a connection there. Then go back to the Cost ETL settings and select it from the Connection dropdown.
      
3. Click Save changes.

Change from Cost ETL to Data Locker connection

Before you begin:

• Consider that after switching to a Data Locker connection, you can't switch back to your previous Cost ETL connection.
• Note that when switching to a Data Locker connection, the folder path structure changes to the following format without impacting the report data: 
  <client-bucket>/<DL-generated-partition>/<connection-name>/t=<report-name>/dt=<YYYY-MM-DD>/v=<1/2/3/4>/

To change from a deprecated Cost ETL connection to a Data Locker connection:

1. In AppsFlyer, from the side menu, go to Export > Cost ETL.
2. In the Data connection section, click Data Locker connections.
     
3. Do one of the following to set up the Data Locker connection:
   • If you already have the data connections you want set up, in the Connection dropdown, select the ones you want for Cost ETL.
      
   • If you don't have the data connection you want set up yet, click + New connection in Data Locker and create a connection there. Then go back to the Cost ETL settings and select it from the Connection dropdown.
      
4. Click Save changes.

In AWS, by default, when AppsFlyer writes objects to your bucket, the object owner is AppsFlyer. Depending on your data loading process, you might have to change the default ownership to you—the bucket owner.  

To change the ownership of objects in your bucket:

1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.
2. In the Buckets list, choose the name of the bucket that you want to enable S3 Object Ownership for.
3. Go to the Permissions tab.
4. Under Object Ownership, click Edit. 
5. Select Bucket owner preferred.
6. Click Save.

The report schedule settings allow you to get the freshest data right when you need it. 

When choosing your report schedule: 

• Select the time to get the report as close as possible to the time you start processing data in your BI system.
• If you have any any ad networks that provide yesterday's data later than others, set an additional report at the time when that ad network data is ready.

When pulling and analyzing your data, it is recommended to pull data for a specific date and batch, or override all previous data for the days that the current batch provides. Otherwise, you may see the same data repeated.

For example, batch 1 on February 20 contains data for Feb 14-20. But, batches written on February 19 also contained data for Feb 14 to Feb 19. Override the data of the previous days received on February 19 with the data received in the most recent February 20 batch.

Not all networks provide data for all dimensions together. Geo and channel data in Meta ads are the most common examples. This is why two separate data sets are provided. The geo data set is guaranteed to have geo data and the channel data set is guaranteed to have channel data.

In many cases and for many media sources, the data in the geo and channel sets will be identical. As such, consume one of the data sets (geo or channel), according to what best suits your needs.

If the integration agreement with a given media source doesn't include channel, so that channel is blank, we treat that data as if it contains the channel. 

Cost ETL provides flexible and granular data as deep as can be extracted from the ad network. To extract actionable insight from such potentially huge amounts of data, it is recommended to aggregate the data in a way that best suits your business needs. For example, if you need to understand cost data at the campaign and country-level, use those dimensions.

Not all networks provide data with the same granularity. For example, Meta ads doesn't provide site ID cost data, while X Ads doesn't provide geo cost data. Be aware of such cases as you aggregate Cost ETL data, and make sure you look at similar data as you compare networks.

Cost ETL provides information regarding all your cost data. Some campaigns provided in Cost ETL do not appear in some AppsFlyer dashboards, for example, data of inactive campaigns, meaning campaigns without any recorded installs. To compare the data, find a specific campaign ID in the overview dashboard and compare it to its cost data in Cost ETL. Learn more about cost data availability

Consider combining Cost ETL reports with Aggregated advanced cohort reports (or regular Cohort reports via Data Locker) in your BI system. Together, they give the fullest picture of marketing performance with fresh and accurate data, including clicks, impressions, cost, revenue, in-app events, etc.). And you can use this combined data to get ROAS, CPA, etc. Learn more