Create S3 Integration

While the CarPurchases collection is being populated with the DynamoDB data, let’s now create a second integration with AWS S3. We will load a Rockset’s Collection with the CSV data from S3. This will allow us to write a JOIN query that combines the car company information with the data from DynamoDB.

1. CREATE AN S3 INTEGRATION

  • Navigate to the left panel and click on Integration. From there, click on Add Integration:

2. USE THE AWS S3 DATA CONNECTOR

  • Click on the AWS S3 data connector and click on Start:

3. GIVE YOUR INTEGRATION A NAME

  • Name your integration CarCompanyInformation.

4. CREATE AN AWS IAM POLICY

  • Copy the bucket name that contains the CSV data from the S3 service . We’re going to use it in the following step. It’ll have this naming pattern: rockset-data-lake-[AccountID]. The AccountID will be a different value for every participant:

  • Navigate to the AWS IAM Policy section and click on Create Policy:

  • Go ahead and override the JSON tab with this JSON snippet:

       {
       "Version": "2012-10-17",
       "Statement": [
         {
           "Effect": "Allow",
           "Action": [
             "s3:List*",
             "s3:GetObject"
           ],
           "Resource": [
             "arn:aws:s3:::rocksetawsworkshop",
             "arn:aws:s3:::rocksetawsworkshop*"
           ]
         }
       ]
     }

    It should look like this:

  • When you’re done, click on Next: Tags.

  • Immediately, click on Next: Review.

  • Set the policy name to RocksetS3Policy and then click Create policy:

5. CREATE AN AWS IAM ROLE

  • Switch back to AWS and navigate to the AWS IAM Role. Click Roles on the left-nav. From there, click Create role:

  • Switch over to the Rockset Console and copy the Rockset Account ID and external ID on the Rockset DynamoDB Integration page:

  • Refer to the image below:
    a) The next section on the integration page is for creating the AWS IAM Role. Click on the AWS account box (2nd red box).
    b) Then click on Another AWS account (which is the 3rd red box).
    c) Put the Rockset Account ID from the Rockset S3 integration page in the text box.
    d) In the Options section, check the Require external ID box.
    e) Paste the external ID from the Rockset S3 integration page in the text box (4th red box).
    f) Click Next on the bottom right of this page.

  • To find the policy we just created in step 4, type RocksetS3Policy in the search bar. Click the policy to attach it to the role. Afterward, click Next:

  • Name the role RocksetS3Role. On the bottom right, click on Create role:

6. RETRIEVING THE ROLE ARN

  • Navigate back to the Roles, and search for the role you just created, RocksetS3Role. Click on the role link:

  • Copy the Role ARN from AWS:

  • Navigate back to Rockset and paste the ARN Role in Role ARN on Rockset and click Save Integration: