AppsFlyer offers a new solution for exporting entire raw data on a daily basis. This data can be processed by customer scripts and imported into their BI systems or be available for pulling when required.
This is a premium feature which is an add-on on top of packages. For specific pricing please contact your Customer Success Manager.
For customers with large amounts of data, Data Locker serves as an alternative to exporting data via Push API. Once the Data Locker is configured, AppsFlyer automatically creates a repository for the data and makes it available for the customer’s convenience.
Benefits of Using Data Locker
- Coverage - the reports can contain data about more than one application
- Simplicity - we manage data storage requirements through AWS
- Reliability - data is stored in AWS which ensures data persistence
- Flexibility - choose what data you want to include in the reports
- More data - get much more data such as Organic Installs and In-App Events, Sessions, Clicks and Impressions
- Accessibility - pull data when required
- Select the Data Locker option under Integrations (this is only available to account owners).
- Select the relevant applications for which data is to be stored. You can choose any number of applications.
- Configure the required data - You can choose to receive Non-Organic, Organic and Retargeting data. The data can include Installs, In-App Events and Uninstalls. In addition, for select customers, we also have the ability to send Sessions, Clicks and Impressions.
- Select the fields you want to include in the report (optional) - Click on the box titled Select Fields to pick the fields required on your data locker reports. Check the box next to any required field to include it in your reports.
If you skip this step or manually select All Fields your reports shall include all possible fields.
NoteAppsFlyer reserves the right to add new fields without prior notice. If your parsing process is not built for field additions, we recommend to manually select the required fields.
- Add recipients (optional)
Add the email or emails (separated by by a comma i.e. firstname.lastname@example.org, email@example.com) of any team member in your account to send them daily notification emails whenever data locker files are generated.
- Click Save.
Once the configuration is saved, a dedicated AWS bucket is created. The bucket is only accessible using customer credentials (for security reasons). In addition, all access to the bucket is audited.
This feature is only visible to account owners.
Data Locker files are based on Raw Data Reports V5 (see: Raw Data Reports V5).
See attached sample file of a CSV file below.
- We run a daily batch job that completes around 9:00 am GMT that populates the bucket with the relevant report.
- Folder structure is: af-ext-raw-data/<Home Folder>/data-locker/<event-type>-<date YYYY-MM-dd>
- The Home Folder is the Home Folder found in the Credential Details window (see screenshot above).
For example for the date 2016-08-12 the relevant report appears under:
** assuming the home folder is support-appsflyer-com
- Each file includes all of the selected apps in the Data Locker configuration. Apps registered to the account at the time of the report job initialization (relating to both Organic and Non-Organic installs).
- The report files under those folders are zipped with gz. (to make the download process efficient at the client end).
- The actual data file is CSV but its name is "part-00000" (with no file extension)
- Any file generated on S3 is available for 30 days.
In each folder, the data is split between many files to avoid files that are too heavy. Depending on the type of data exported, each folder contains between 50 to 1000 files (this number can change without notice). Files are named `part-00000`, `part-00001`, etcEach file has a header row.In each folder, after the files are successfully generated, an empty file called `_SUCCESS` is generated. We recommend checking if this file exists before processing the data.The separator is the coma `,` ; values containing a coma are contained between double quotes `"`, example
`"iPhone6,1"`Each folder is available for 30 days, after which it is deleted.
Accessing the Data
The data can be accessed in two different ways:
AppsFlyer generates an AWS principle (ARN in Amazon terms) under our account and generates AWSKey and AWSSecret credentials for that principle. We then set a policy to allow that principle to both browse and retrieve files from the bucket. Once you retrieve the credentials from the Credential Details section you can access the bucket using any AWS command line tool as well as most FTP clients.
The AWSKey and AWSSecret are found in the Data Locker Configuration screen. See the Setup section for more information.
We recommend using AWS CLI (Command Line Interface) on Mac and Linux systems for accessing the data. On windows we recommend using Amazon's S3 Browser, but not DragonDisk, which is not fully supported.
For AWS Cli Tools, perform the following steps:
Open Terminal at your HOME folder (use the Bucket Name as the Home Folder):
- Enter aws configure
- Enter your AWS Access Key
- Enter your AWS Secret Key
- Enter eu-west-1
- Click Enter (None)
To list your folders under your bucket:
aws s3 ls s3://af-ext-raw-data/<home-folder>/data-locker/
To list files under a specific folder:
aws s3 ls s3://af-ext-raw-data/<home-folder>/data-locker/<event-type>-<YYYY-MM-dd>/
** valid for LINUX base OS
For each file, the following information is available:
|clicks||Clicks reports. This type of report is only available in Data Locker.||No||Yes|
|impressions||Impressions. This type of report is only available in Data Locker.||No||Yes|
|inapp||In-App Events report. Contains data about both organic and non-organic in-app events.||Yes||Yes|
|inappreattrReport||In-App Events resulting from re-attributions.||No||Yes|
|inappretargets||In-App Events resulting from re-engagement.||No||Yes|
|installs||Installs report. Contains data about for both organic and non-organic installs.||Yes||Yes|
|launches||App sessions. This type of report is only available in Data Locker.||Yes||No|
Using Reports as Data Sources
You can use the data from the reports and add it to your own databases. To extract the data and add it to your databases you need to know the report format. The report format depends on the fields that you choose to include.
If you want to use reports from data locker as data sources, we recommend using an existing report as a sample. Existing reports give you an accurate sense of the report format. Knowing the report format in advance makes it easier to implement ETL for data extraction purposes.
Many reports contain valuable data that you can use for campaign optimization and retargeting.
- Clicks report - the clicks report contains the IDFA or Google Advertising ID. You can use these IDs to retarget users that click on ads but don't engage with the app.
- Impression report - Like the clicks report, the impression report also contains the IDFA or Google Advertising ID. You can use the impression report to optimize campaigns according to impressions that didn't lead to clicks. You can also retarget these users with different ads and in different campaigns.
- Retargeting and Re-attribution report - these reports also contain the IDFA or Google Advertising ID. You can use the IDFA or Google Advertising ID to highlight those users that you managed to retarget. Knowing what users you managed to retarget can help you optimize retargeting campaigns.
Note: To benefit from IDFA or Google Advertising ID as explained above, add them to tracking links.
Data Locker has some limitations in terms of data accessibility and formatting.
Amazon Clicks and Impressions - Amazon clicks and impressions are not supported and do not appear in the reports that are stored in Data Locker.
Timezone and Currency - Timezone and currency in Data Locker are always UTC and USD respectively. No matter what timezone and currency you set in app settings, Data Locker reports always display the timezone and currency in UTC and USD.