Data Collaboration Platform (DCP)—Set up your cloud services

Last update:

At a glance: Set up your data warehouses, such as BigQuery and Snowflake, and your cloud storage buckets, such as Amazon S3 or GCS to connect them to the DCP. You'll then be able to share your data and access audiences created in the DCP.

About DCP

The Data Collaboration Platform (DCP) functions as the central point for data collaboration, including audience creation and activation. DCP relies on the advanced technology of the Data Clean Room (DCR) to ensure data privacy and security for the collaboration and audience management processes.

Overview

Employing the Data Collaboration Platform (DCP) involves establishing inbound and outbound connections between your cloud services and the DCP:

  • Inbound connections are used to access your first-party data when creating sources.
  • Outbound connections serve as destinations for audiences created within the platform.

Creating these connections is a 2-step process:

 Note

See source data requirements for guidance on the following source data requirements:

  • Data format (for all sources)
  • Table columns (for sources in data warehouses)
  • File name and format (for sources in cloud storage buckets)

Supported cloud services

Two types of cloud services are supported for inbound and outbound connections to the DCR:

  • Data warehouses: BigQuery, Snowflake, and Salesforce Cloud/CRM
  • Cloud storage buckets: Amazon S3 (AWS) and GCS

 Important!

When using the same cloud storage bucket on Amazon S3 or GCS for both inbound and outbound connections, be sure to follow the special instructions for that setup.

Setting up cloud services and warehouses for inbound connections

Prepare your selected cloud services and warehouses for use with DCR inbound connections according to the instructions in the following tabs.

 Note

When completing this stage, make sure to copy the cloud platform path of your cloud service or warehouse. You'll need it when adding connections.

BigQuery, Snowflake, Amazon S3, and GCS

BigQuery

Note: The following procedure must be performed by your Google Cloud admin.

Create a dataset and grant AppsFlyer permissions.

To create a dataset: 

  1. Log in to your Google Cloud console.
  2. Go to the BigQuery page.
  3. In a new or existing Google Cloud project, create a dataset for the exclusive use of the DCR:
    1. In the left-side panel, click the three-dot menu icon (View actions) to the right of the project ID.
    2. Select Create dataset.

      BQ_create_dataset.png

    3. In the right-side panel that opens, enter the name of the dataset and select other options as you require.
      • You can use any name that suits you – using letters, numbers, and underscores (_) only.
        • Recommended: Use a name that indicates the dataset is being used for an inbound connection.
      • It is strongly recommended NOT to use the Enable table expiration option since the DCR would be unable to read the sources after the tables expire.

    4. Click CREAT DATASET.

To grant AppsFlyer permissions to the dataset:

    1. In the left-side panel, click the three-dot menu icon (View actions) to the right of the dataset you created.
    2. Select Share.
    3. In the right-side panel that opens, click ADD PRINCIPAL.
    4. In the Add principals section, enter the following account in the New principals field:
      appsflyer-dcr@dcr-report.iam.gserviceaccount.com
    5. In the Assign roles section, select BigQuery > BigQuery Data Viewer.

      BQ_data_viewer.png

    6. Click Save.
    7. Click CLOSE to close the right-side panel.

Snowflake

Note: The following procedure must be performed by a Snowflake Accountadmin.

To create a private share for use by the DCR:

  1. Log in to the Snowflake account that contains the data you want to share with the DCR.
  2. Switch your role to Accountadmin.
  3. From the left-side panel, select Private Sharing.
  4. On the page that opens, select the Shared By Your Account tab.

    snowflake_private_sharing.png

  5. Click Share. From the list that opens, select Create a Direct Share.
  6. Select the tables and/or views that you want to share with the DCR, then click Done.
  7. According to your needs, change the Secure Share Identifier and add an optional description.
  8. In the field Add accounts in your region by name, enter one of the following AppsFlyer Snowflake accounts, according to your Snowflake account region:
    Region AppsFlyer account
    EU West (eu-west-1) QL63117
    US East - N. Virginia (us-east-1) MWB70410
    US East - Ohio (us-east-2) BM15378
  9. Click Create Share. 

Amazon S3

 Note

Create a bucket and grant AppsFlyer permissions. 

To create a bucket: 

  1. Log in to the AWS console.
  2. Go to the S3 service.
  3. Create the bucket:
    1. Click Create bucket.
    2. Complete the Bucket name, starting with af-dcr- or af-datalocker- and followed by your text (according to the DCR naming requirements above).
    3. Click Create bucket.

To grant AppsFlyer bucket permissions:

    1. Select the bucket you created. 
    2. Go to the Permissions tab. 
    3. In the Bucket policy section, click Edit.
      The Edit bucket policy window opens.
    4. Paste the following code snippet into the window. 
      {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AF-DCR-DL",
      "Effect": "Allow",
      "Principal": {
        "AWS": [         "arn:aws:iam::195229424603:user/product=dcr-reporter__envtype=prod__ns=default",   "arn:aws:iam::195229424603:user/product=datalocker__envtype=prod__ns=default"
        ]
      },
      "Action": [
        "s3:GetObject",
        "s3:ListBucket",
        "s3:DeleteObject",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::af-dcr-mybucket",
        "arn:aws:s3:::af-dcr-mybucket/*"
      ]
    }
  ]
}

  2. In the snippet, replace af-dcr-mybucket (in the 2 lines in which it appears) with the name of the bucket you created.
    Caution! When replacing the bucket name in the snippet, be sure not to overwrite /* in the second line in which the bucket name appears.

  3. Click Save changes.

GCS

 Note

Create a bucket and grant AppsFlyer permissions. 

To create a bucket: 

  1. Log in to your GCS console.
  2. Go to the Cloud Storage Browser page.
  3. Create the bucket:
    1. Click Create bucket.
    2. Enter your bucket information on the Create a bucket page. Include the bucket name, starting with af-dcr- or af-datalocker- and followed by your text (according to the DCR naming requirements above).
    3. Click Continue.
    4. Click Create.

To grant AppsFlyer bucket permissions:

    1. Select the bucket you created. 
    2. Go to the Permissions tab. 
    3. In the Permissions section, click + Add.
      The Add members window opens.
    4. In the New members box, enter the following account:
      appsflyer-dcr@dcr-report.iam.gserviceaccount.com
    5. From the Role list, select Cloud storage > Storage Admin.
    6. Click Save.

Configuration guidelines for Amazon S3 and GCS

You can use one or more buckets for uploading data to the DCR (on Amazon S3, GCS, or both). However, in most cases, the easiest-to-manage structure includes a single bucket on a single cloud service.

  • You can set up the same bucket for use with both inbound and outbound connections by following these instructions.

The following requirements are relevant to buckets on both cloud services:

  • Use: The bucket must be for the exclusive use of AppsFlyer Data Clean Room. In other words, no other service can write data to the bucket.
  • Permissions: AppsFlyer DCR service must be given bucket permissions. See instructions for granting these permissions in the tabs for each cloud service below.
  • Name: The bucket name must begin with af-dcr- or af-datalocker-
    • Example: af-dcr-example-bucket
  • DCR naming requirements: see details below.

DCR naming requirements

The following naming requirements apply to all DCR data entities (buckets, folders, and files):

  • Maximum length: 200 characters
  • Valid characters:
    • letters (A-Z, a-z)
    • numbers (0-9), cannot be the first character of a name
    • hyphens (-), cannot be the first character of a name
    • Invalid characters:
      • spaces
      • all other symbols or special characters
    • Characters used for special purposes only:

Setting up cloud services for outbound connections

The DCR exports the audiences you've created to your selected cloud services using AppsFlyer Data Locker.

  • Note: Receiving DCR audiences using the AppsFlyer Data Locker doesn't require a premium subscription to Data Locker. However, if you're interested in receiving other AppsFlyer reports via Data Locker, contact your CSM or send an email to hello@appsflyer.com.

Your audiences can be sent to one or more locations on your cloud services (whether or not you use the same services for inbound connections). Prepare them for use with outbound connections according to the instructions in the following tabs.

Data warehouses – BigQuery and Snowflake

BigQuery

Note: The following procedure must be performed by your Google Cloud admin.

Create a dataset and grant Data Locker permissions.

To create a dataset: 

  1. Log in to your Google Cloud console.
  2. Go to the BigQuery page.
  3. In a new or existing Google Cloud project, create a dataset for the exclusive use of Data Locker:
    1. In the left-side panel, click the three-dots menu icon (View actions) to the right of the project ID.
    2. Select Create dataset.

      BQ_create_dataset.png

    3. In the right-side panel that opens, enter the name of the dataset and select other options as you require.
      • You can use any name that suits you – using letters, numbers, and underscores (_) only.
        • Recommended: Use a name that indicates the dataset is being used for an outbound connection.
      • It is strongly recommended NOT to use the Enable table expiration option since Data Locker would be unable to send your audiences to the dataset after the tables expire.

    4. Click CREATE DATASET.

To grant Data Locker permissions to the dataset:

    1. In the left-side panel, click the View actions button BQ_view_actions_button.png to the right of the dataset you created.
    2. Select Share.
    3. In the right-side panel that opens, click ADD PRINCIPAL button.
    4. In the Add principals section, enter the following account in the New principals field:
      datalocker-bq-admin-prod@datalocker-bq-prod.iam.gserviceaccount.com
    5. In the Assign roles section, select BigQuery > BigQuery Data Editor.

      BQ_data_editor.png

    6. Click Save.
    7. Click CLOSE to close the right-side panel.

Snowflake

The procedure for preparing Snowflake for outbound connections is completed in combination with the procedure for creating the outbound connection itself.

Cloud storage buckets – Amazon S3 and GCS

The procedure for preparing cloud storage buckets for outbound connections is very similar to the one preparing them for inbound connections (including the instructions relevant to both cloud storage services).

The instructions in the tabs below apply when you are using a bucket for outbound connections only.

Amazon S3

Follow the instructions for creating an Amazon S3 bucket for inbound connections (with no changes to that procedure).

GCS

Follow the instructions for creating a GCS bucket for inbound connections. In step #4 of that procedure, enter the following account in the New members box:
af-data-delivery@af-raw-data.iam.gserviceaccount.com

Setting up the same cloud storage bucket for both inbound and outbound connections

As previously mentioned, it's common to use the same bucket on Amazon S3 or GCS for both inbound and outbound connections.

The instructions for this setup vary only slightly from the instructions for inbound connections. They do differ, however, depending on whether you are: 

  • creating a new bucket for use with DCR inbound and outbound connections; or
  • modifying a bucket previously used only for Data Locker to one now used for both inbound and outbound DCR connections

Instructions for both of these scenarios are included in the tabs below:

Amazon S3

Creating a new bucket for inbound/outbound connections

Follow the instructions for creating an Amazon S3 bucket for inbound connections (with no changes to that procedure).

Modifying an existing bucket previously used only for Data Locker

Modifying an existing bucket that you used previously only for Data Locker requires changing bucket permissions (to allow access by both DCR and Data Locker).

To modify bucket permissions:

  1. Log in to the AWS console.
  2. Go to the S3 service.
  3. Select the bucket used previously only for Data Locker. 
  4. Go to the Permissions tab. 
  5. In the Bucket policy section, click Edit.
    The Edit bucket policy window opens.
  6. Replace the contents of the window with following code snippet: 
    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AF-DCR-DL",
      "Effect": "Allow",
      "Principal": {
        "AWS": [         "arn:aws:iam::195229424603:user/product=dcr-reporter__envtype=prod__ns=default",   "arn:aws:iam::195229424603:user/product=datalocker__envtype=prod__ns=default"
        ]
      },
      "Action": [
        "s3:GetObject",
        "s3:ListBucket",
        "s3:DeleteObject",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::af-dcr-mybucket",
        "arn:aws:s3:::af-dcr-mybucket/*"
      ]
    }
  ]
}
    • In the snippet, replace af-dcr-mybucket (in the 2 lines in which it appears) with the name of the bucket you created.
    • Caution! When replacing the bucket name in the snippet, be sure not to overwrite /* in the second line in which the bucket name appears.
  7. Click Save changes.

GCS

Creating a new bucket for inbound/outbound connections

Follow the instructions for creating a GCS bucket for inbound connections. Modify step #4 of that procedure to enter the following 2 accounts in the New members box:
appsflyer-dcr@dcr-report.iam.gserviceaccount.com
af-data-delivery@af-raw-data.iam.gserviceaccount.com

Modifying an existing bucket previously used only for Data Locker

Modifying an existing bucket that you used previously only for Data Locker requires changing bucket permissions (to allow access by both DCR and Data Locker).

To modify bucket permissions:

  1. Log in to your GCS console.
  2. Go to the Cloud Storage Browser page.
  3. Select the bucket used previously only for Data Locker. 
  4. Go to the Permissions tab.
  5. In the Permissions section, click + Add.
    The Add members window opens.
  6. In the New members box, enter the following account:
    appsflyer-dcr@dcr-report.iam.gserviceaccount.com
  7. From the Role list, select Cloud storage > Storage Admin.
  8. Click Save.