Data Clean Room—Working with sources

At a glance: Set up the data sources you upload to the Data Clean Room (DCR) for enrichment with AppsFlyer attribution data and DCR report creation.

Before you begin: Read the article Data Clean Room—cloud storage and data file setup to understand cloud storage setup and source file requirements.

Introduction

After you have created a source data file, the next step is to set it up as a source in the AppsFlyer DCR platform. Defining a source consists of 2 main elements:

You can see a list of all existing sources in the Sources tab of the Data Clean Room.

Source location

The source location consists of a cloud storage bucket (known as a connector) and the underlying file path from which the DCR will read the source file each time it is updated. 

This bucket and file path are created using the interface of your selected cloud service. The full instructions for this process, including examples, are detailed here.

Connectors

A connector identifies the cloud service bucket (on either AWS or GCS) in which DCR data is stored. This is the location from which source data is uploaded and to which reports are delivered.

  • Generally, it's easiest to use the same bucket for all the data you upload to and all the reports you receive from the DCR. However, you can use any combination buckets and cloud services as long as you define each as a connector in the AppsFlyer DCR platform.

Follow these steps to define a connector:

  1. From the main page of the Data Clean Room, click Manage connectors in the top-right corner.
  2. In the New connector dialog, enter the bucket name exactly you created it on your cloud service.

What if I don't know the source location?

It's best practice to specify the source location before defining the source structure; however, this may not be possible in all cases (for example, if your cloud storage structure is not yet finalized). If you want to proceed with defining the source structure before identifying the source location, you can upload a prototype source file from your computer (local file).

 Important!

A source cannot be saved until a source location is specified.

To save the source before you know its location, you can save it using a temporary (even non-working) source location. However, in order for the DCR to begin processing a report that uses this source, you must provide the source's actual (working) location by editing it later.

Source structure

For all sources that you upload to the DCR for processing, AppsFlyer needs to know how each data field should be used for purposes of creating reports. Defining the source structure consists of loading a source file and categorizing each field as one of the following types:

  • Identifier: Field that identifies a unique app user (examples might include CUID, AppsFlyer ID, etc.)
    • The primary purpose of identifiers in the context of the DCR is to join data sources so that corresponding user-level data can be matched.
  • Dimension: An attribute by which you categorize app users (examples might include geo, install date, campaign, etc.)
  • Metric: Numeric data you have collected with respect to an app user (examples might include revenue, number of app opens, LTV, etc.)
    • A data field identified as a metric can contain only numeric values.

Creating a source

The process of creating a source consists of all the steps described below. They are separated into tabs simply for ease of reading.

Preliminary steps

Follow these steps to create a source:

  1. Go to the Sources tab of the Data Clean Room.
  2. Click the + New source button.
    The New source page opens.
  3. Enter the name of the source.
    • This can be any unique name that will help you identify the source within the AppsFlyer DCR platform. It does not need to match the file name.
    • Important! Ensure that the source name is different all other sources in your account or you will not be able to save the source.
    • Source name requirements:
      • Length: 2-80 characters
      • Valid characters:
        • letters (A-Z, a-z)
        • numbers (0-9), cannot be the first character of a name
      • Invalid characters:
        • spaces
        • all other symbols or special characters
  4. Follow the relevant instructions below based on where AppsFlyer should read the source file in order to load its fields.

Loading fields from the source location

To load fields from the source location:

  1. Select Load fields from source location.
    The Source location and format dialog will open.
  2. Select the connector in which the source file path is located.
    • If there are no connectors defined in your account, the New connector dialog will open, prompting you to create one.
    • If you have existing connectors but want to use a new one, click the + New connector button in the top right of the Source location and format dialog.
  3. Enter the source folder path:
    • The bucket and DCR key folder will display automatically.
    • Enter the full path following the DCR key folder, up to and including the second-level data source folder.
    • Your source folder path might look something like this:
      s3://af-dcr-example-bucket/01bcc5fb/input/BI-data/
  4. Select the format in which source files will be uploaded.
  5. Click Apply.

Loading fields from a local file

To load fields from a local file:

  1. Select Load fields from local file.
  2. The File upload dialog will open.
  3. Click Click to upload, and select the prototype source file.
  4. Click Apply.

Defining the source structure

Follow these steps to define the source structure:

  1. After you complete the steps above (loading fields either from the source location or from a local file), AppsFlyer analyzes the file, and a list of all data fields (columns) is displayed in the Available fields list.
  2. Select one or more of the fields in the Available fields list on the left and use the buttons in the middle of the screen to categorize them as identifiers, dimensions, or metrics.
    • Once you categorize a field, it is displayed in the relevant category list on the right side of the screen.
    • You can use the search bar to search for fields in the lists.
  3. You can remove a field from a category it's been previously assigned to by selecting it in the relevant category list and using the Remove button to return it to the Available fields list.
  4. Repeat this process until you have categorized each field you want to include in DCR reports.
    • There is no requirement to categorize every field in the Available fields list. However, a field must be categorized in order to use it later in a report.
    • If you decide later that want to use additional fields from this source, you can do so by editing the source structure. Note, however, that any fields not previously categorized do not load automatically into the Available fields list. To work with these fields, you must reload them from the source location or a local file.

Saving the source

 Note

In order to save a source, you must specify a source location. Even if you're not yet able to specify a permanent location, continue with the steps below for If you loaded fields from a local file, but provide a temporary location instead of a permanent one. The temporary location does not need to be a working location. Important! When you want to start uploading data so that the DCR will begin processing reports that use this source, follow the steps for editing the source to provide its permanent location.

Follow these steps to save the source:

  1. If you loaded fields from the source location, click Save.
  2. If you loaded fields from a local file:
    1. Click Set source location.
      The Source location and format dialog will open.
    2. Select the connector on which the source file path is located.
      1. If there are no connectors defined in your account, the New connector dialog will open, prompting you to create one.
      2. If you want to use a new connector for this source, click the + New connector button.
    3. Enter the source folder path:
      1. The bucket and DCR key folder will display automatically.
      2. Enter the full path following the DCR key folder, up to and including the second-level data source folder.
      3. Your source folder path might look something like this:
        s3://af-dcr-example-bucket/01bcc5fb/input/BI-data/
    4. Select the format in which source files will be uploaded.
      • Important! If the file format selected here does not match the actual file format of the source files uploaded, DCR processing will fail.
    5. Click Apply to apply the source location.
    6. Click Save to save the source.
    7. The new source will now display in the Sources tab of the Data Clean Room.

Editing or deleting a source

There are several ways in which you can edit a source (or delete it):

  • Editing the source name
  • Editing the source location or file format
  • Editing the source structure
  • Deleting a source (if it's not being used for a report)

Editing the source name

To edit the source name:

  1. Go to the Sources tab of the Data Clean Room.
  2. In the list of sources, hover over the row of the source you want to edit.
  3. Click the edit button edit_button.png that displays on the right side of the row.
  4. On the Edit source page, edit the name of the source.
  5. Click the Update button to save the source with the new name or Discard changes to undo your changes.

Editing the source location or file format

To edit the source location:

  1. Go to the Sources tab of the Data Clean Room.
  2. In the list of sources, hover over the row of the source you want to edit.
  3. Click the edit button edit_button.png that displays on the right side of the row.
  4. On the Edit source page, scroll down to the Source location section.
  5. Click the edit button edit_button.png next to the current source location.
  6. Make the necessary changes in the Source location and format dialog.
  7. Click Apply to implement your changes.
  8. Click the Update button to save the source with the new location/file format or Discard changes to undo your changes.

Editing the source structure

To edit the source structure:

  1. Go to the Sources tab of the Data Clean Room.
  2. In the list of sources, hover over the row of the source you want to edit.
  3. Click the edit button edit_button.png that displays on the right side of the row.
  4. On the Edit source page, the fields that were previously categorized as identifiers, dimensions, or metrics will display in the relevant category lists on the right side of the screen.
  5. You can move a previously categorized field to a different category without reloading fields from the source file. To do this:
    1. First, select it in the relevant category list and use the Remove button to return it to the Available fields list.
    2. Next, select it in the Available fields list and use the buttons in the middle of the screen to categorize it as an identifier, dimension, or metric.
  6. To work with fields in the source file that have not yet been categorized, they must be reloaded from the source location or from a local file. Make this selection by clicking the relevant button at the bottom of the Available fields list.
  7. AppsFlyer analyzes the file, and a list of all previously uncategorized data fields (columns) is displayed in the Available fields list.
    • Fields that were previously categorized as identifiers, dimensions, or metrics will still display in the relevant category lists on the right side of the screen.
    • If a previously categorized field is not found in the reloaded source file, it will still display in the relevant category list, but it will be marked with an error icon.
  8. Categorize new fields by selecting one or more of them in the Available fields list on the left and using the buttons in the middle of the screen to categorize them as identifiers, dimensions, or metrics.
  9. Once you have made all necessary changes, click the Update button to save the source with the updated structure or Discard changes to undo your changes.

 Important!

Don't forget to make corresponding changes reflecting the new source structure in any reports for which this source is used:

  • Fields that were removed, uncategorized, or changed from their previous categories will be automatically removed from any reports in which are used.
  • Newly added or categorized fields will not be automatically included in existing reports until you edit report definitions to include them.

Deleting a source

  1. Go to the Sources tab of the Data Clean Room.
  2. In the list of sources, hover over the row of the source you want to delete.
  3. Click the delete button delete_button.png that displays on the right side of the row.
  4. In the dialog, confirm that you want to delete the source.
    • You cannot delete a source that is being used by a report. If this is the case, a message will list the reports in which the source is being used. In order to delete the source, you can either:
      • Delete the reports in which it is being used; or
      • Remove the source fields from the definitions of the reports in which they are used.
Was this article helpful?