Data Locker is a robust solution for storing and extracting raw data, offering both rich data and the ease of obtaining it.
For customers with large amounts of data, Data Locker is an alternative to exporting data using Pull API. Customers can choose to view and extract raw data by report type, days and hours. You can use automated scripts to pull and process the data, import it into your BI systems or make it available on demand.
This is a premium feature. For pricing information contact your Customer Success Manager.
Click here for information about selecting the best data APIs for you
Benefits of using Data Locker
- Coverage: the reports can contain data about more than one application
- Simplicity: we manage data storage requirements through AWS
- Reliability: data is stored in AWS which ensures data persistence
- Flexibility: choose what data you want to include in the reports
- Granularity: data is segmented into report types, days and hours
- Unique data: get more data such as Organic Installs, In-App Events, Sessions, Clicks, and Impressions
- Accessibility: pull data when required
Data in Data Locker is segmented into reports, days and hours. For any type of report for a given day, AppsFlyer separates each hour of activity into its own folder.
For example, the folder path
/t=installs/dt=2019-01-17/ contains 25 folders. There are 24 folders for each hour of the day and another folder called Late.
- To understand the folder structure and how hourly segmentation works, see listing files and folders.
- For an explanation on the Late folder, click here.
In Data Locker AppsFlyer separates data into hourly folders, but this data is not logged in real-time. To learn more, click here.
Configuring Data Locker
- Select the Data Locker option under Integrations (this is only available to account owners).
- Select the applications that you want to include in the reports. You can choose any number of applications. Once you select the apps, click Apply.
- Optional - select the media sources that you want to include in the reports. You can only select media sources after selecting apps. Once you select the media sources, click Apply. If you skip this step, AppsFlyer includes ALL media sources in the reports.
- Select the report types that you want to receive - you can select Acquisition, Retargeting and Protect 360 data.
- Acquisition - Clicks, Impressions, Installs, In-App Events, Sessions, Uninstalls
- Retargeting- Retargeting Clicks, Retargeting Conversions, Retargeting In-App Events
- Protect 360- Blocked Installs, Blocked In-App Events, Blocked Clicks
- People Based Attribution - Web Conversions (if People-Based Attribution is enabled in your account). For People-Based Attribution, data locker aggregates data daily in one report. This report can be found in Data Locker in the 23rd-hour folder. For example t=web_touch_points/dt=2019-07-19/h=23.
Protect 360 reports are available only to clients who have Protect 360 enabled.
- Optional - select the In-App Events that you want to include in the reports. Once you select the events, click Apply. If you skip this step, AppsFlyer includes ALL in-app events in the reports.
- Optional - Click on the box titled Fields to select the fields you want to include in the reports. Check the box next to any required field to include it in your reports. If you skip this step or manually Select All, your reports include all possible fields.
Important!AppsFlyer reserves the right to add new fields without prior notice. If your parsing process is not built for field additions, we recommend to manually select the required fields.
- Optional - Add recipients to notify them when data locker files are generated.
To add more than one recipient, add the emails separated by a comma i.e. firstname.lastname@example.org, email@example.com.
- Click Create Bucket.
Once the configuration is saved, a dedicated AWS bucket is created. The bucket details appear at the top right-hand corner of the screen. They include the Bucket Name, Home Folder, and credentials for accessing data.
The bucket is only accessible using customer credentials (for security reasons). In addition, all access to the bucket is audited.
The configuration above is for AppsFlyer Data Locker 2.0. Instructions for Data Locker 1.0 can be found here.
Permission to configure Data Locker
- Only account admins can configure Data Locker.
- We run an hourly batch job that populates the bucket with the relevant report.
- Each file includes all of the selected apps in the Data Locker configuration.
- Any file generated on S3 is available for 30 days.
- Each folder is available for 30 days.
- Data (files and folders) is deleted after 30 days.
Folder structure and format
- Folder structure is: af-ext-reports/<Home Folder>/data-locker-hourly/t=<event-type>/dt=<date YYYY-MM-dd>/h=<Hour h>
- The Home Folder is the Home Folder that appears in the Credentials window (see step 8 in the setup instruction above).
- For example, for the date 2016-08-12 the relevant report appears under:
In each folder, the data is split between many files to avoid files that are too heavy. Depending on the type of data exported, each folder contains between 50 to 1000 files (this number can change without notice). Files are named `part-00000`, `part-00001`, etc.
In each folder, after the files are successfully generated, an empty file called `_SUCCESS` is generated. We recommend checking that this file exists before processing the data.
File structure and format
- Data Locker files are based on Raw Data Reports V5 (see: Raw Data Reports V5).
- The actual data file is in CSV format but it has no file extension.
- The report files are zipped in .gz format (to make the download process efficient).
- Each file has a header row.
- Values that have a coma in them are contained between double quotes `"`, for example
Accessing the data
AppsFlyer creates an AWS principle (ARN in Amazon terms) and generates credentials for that principle. We then set a policy to allow that principle to both browse and retrieve files from the bucket.
You can access the bucket using any AWS command line tool as well as most FTP clients. Make sure to retrieve the credentials (AWS Access Key and AWS Secret) from the Credentials section.
The data can be accessed in two ways:
We recommend using AWS CLI (Command Line Interface) on Mac and Linux systems for accessing the data. On windows we recommend using Amazon's S3 Browser, but not DragonDisk, which is not fully supported.
Amazon's S3 browser
An AWS GUI Client example:
For AWS CLI Tools, perform the following steps:
Open Terminal at your HOME folder (use the Bucket Name as the Home Folder):
- Enter aws configure
- Enter your AWS Access Key
- Enter your AWS Secret Key
- Enter eu-west-1
- Click Enter (None)
Listing folders in your bucket
aws s3 ls s3://af-ext-reports/<home-folder>/data-locker-hourly/
Listing files and folders
There are three types of folders in your Data Locker bucket:
- Report Type
To list all the reports of a specific report type:
aws s3 ls s3://af-ext-reports/<home-folder>/data-locker-hourly/t=installs/
To list all the reports of a specific report type for a specific day:
aws s3 ls s3://af-ext-reports/<home-folder>/data-locker-hourly/t=installs/dt=2019-01-17
To list all the reports of a specific report, in a specific hour of a specific day:
aws s3 ls s3://af-ext-reports/<home-folder>/data-locker-hourly/t=installs/dt=2019-01-17/h=23
The Late folder is the 25th folder in a given day folder.
This folder answers the need to log events that arrive when certain data processing takes place.
If an event is sent to AppsFlyer on January 21st between 12 AM and 2 AM and the event has a timestamp of January 20th, this event is populated in the Late folder inside the
For each file, the following information is available:
|clicks||Clicks reports. This type of report is only available in Data Locker.||No||Yes|
|clicks_retargeting||Click reports for clicks coming from retargeting campaigns. This type of report is only available in Data Locker.||No||Yes|
|impressions||Impressions. This type of report is only available in Data Locker.||No||Yes|
|installs||Installs reports. Contains data about both organic and non-organic installs.||Yes||Yes|
|inapps||In-App Events report. Contains data about both organic and non-organic in-app events.||Yes||Yes|
|conversions_retargeting||Retargeting reports for both re-engagements and re-attributions||No||Yes|
|inapp_retargeting||In-App Events resulting from re-attributions and re-engagements||No||Yes|
|sessions||App sessions. This type of report is only available in Data Locker.||Yes||Yes|
|blocked_installs||Protect360 blocked installs.||Yes|
|blocked_inapps||Protect360 blocked in-app events.||Yes|
|blocked_clicks||Protect360 blocked clicks.||Yes|
Using reports as data sources
You can use the data from the reports and add it to your own databases. To extract the data and add it to your databases you need to know the report format. Data Locker reports are based on Raw Data Reports. However, the final report format depends on the fields that you choose to include.
The report format for reports in Data Locker includes all the fields that are available in Raw Data V5 reports.
There is one main difference between Raw Data V5 reports and Data Locker reports. Data Locker reports contain all fields that are available in raw data reports. For example, a V5 installs report contains 90 fields. A Data Locker installs report contains 100 fields. Click here to download a sample file that you can use to compare the two reports.
Many reports contain valuable data that you can use for campaign optimization and retargeting.
- Clicks report - the clicks report contains the IDFA or Google Advertising ID. You can use these IDs to retarget users that engage with your ads but fail to install the app.
- Impression report - Like the clicks report, the impression report also contains the IDFA or Google Advertising ID. You can use the impression report to optimize campaigns according to impressions that don't lead to clicks. You can also retarget these users with different ads and in different campaigns.
- Retargeting and Re-attribution report - these reports also contain the IDFA or Google Advertising ID. You can use the IDFA or Google Advertising ID to highlight those users that you manage to retarget. Knowing what users you manage to retarget can help you optimize retargeting campaigns.
Note: To benefit from IDFA or Google Advertising ID as explained above, make sure they are included in all your attribution links.
AppsFlyer data locker separates data into hourly folders. The time it takes to create hourly folders depends on the amount of data that needs processing. Therefore, data is not processed and logged in real time.
AppsFlyer receives data for activity between 14 PM and 15 PM on January 17, 2019. At 15 PM AppsFlyer starts processing the data.
Since it takes some time for AppsFlyer to fully process the data, reports are not available on the hour. So the report in the folder
/t=installs/dt=2019-17-01/h=14 is not available on January 17th, 2019 at 15 PM.
Creating Hourly Folders
If there is no data for a specific hour, Data Locker still creates a folder for it. This might lead to a situation where automated processes break because a folder exists but is empty.
Always design your data retrieval processes such that they can handle empty hourly folders.
SRNs clicks and impressions
Over 99% of integrated ad networks use AppsFlyer's attribution links for clicks and impressions attribution. For these networks, Data Locker provides the full engagements data.
However, Self-reporting networks (SRNs) only share the engagement data of clicks and impressions that result in an install.
Amazon clicks and impressions
Amazon clicks and impressions are not supported. They do not appear in the reports that are stored in Data Locker.
Timezone and currency
Timezone and currency in app settings don't have an effect on data in Data Locker. Data Locker reports always display the timezone and currency in UTC and USD.