If your organization uses Google Cloud Storage (GCS), you can transfer data from Data Locker to GCS.
One of the major benefits of transferring data from Data Locker to GCS is that you can import data to Google Big Query. You don't need to rely on your organization's developers to retrieve data from Data Locker. With a simple UI configuration you can transfer the data straight into Google Big Query.
Google Big Query can easily handle data from Data Locker thanks to the standard structure of the CSV files in Data Locker. Once you import data into Google Big Query you can start analyzing it right away.
Before you can configure GCS to pull data from Data Locker, the following is required:
AWS Access Key and Secret
- Access the app's dashboard
- In the left-hand side menu, click on Data Locker under Integration
- Look for Credentials Details in the right-hand side of the screen
- Copy your AWS Access Key
- Click on Show Secret
- Copy your secret key
Your Data Locker Home Folder
In the Data Locker screen, under Credentials Details, copy your Data Locker Home Folder.
Configuring a Transfer Job in GCS
The first step is to configure a transfer job in GCS. Here, you configure GCS to pull data from Data Locker on a daily basis. Once the data gets into GCS, you can import it to Big Query.
- Open your GCS Transfer Console
- Click on Transfer in the left-hand side menu
- Click on Create transfer
- Under Select Source choose Amazon S3 bucket
- In the Amazon S3 bucket field put af-ext-reports
- In the Transfer files with these prefixes field put the following path: your-home-folder/data-locker-hourly/
- Specify your access key and secret in the relevant fields
- Under Select Destination specify the GCS storage bucket
- Under Configure Transfer choose when to run the transfer job
Importing Data from GCS to Big Query
Once you have data from Data Locker imported into GCS, you can transfer it to Big Query. The files are zipped in .gz format but Big Query knows how to unzip the files. Big Query also handles the standard CSV format of files from Data Locker for seamless table creation. Once you can create a table, you can go ahead and analyze your data quickly and easily.