To ingest assets from your AWS S3 bucket into the Coactive platform, you first need to establish a secure connection that allows Coactive to access your data. There are two supported methods:

1. Using an IAM role, where you create a role in your AWS account that Coactive can temporarily assume, offering fine-grained control and audibility.
2. Using a bucket policy, where you directly grant Coactive’s role permission to access your bucket without needing to manage IAM roles.

Both methods ensure secure, read-only access and are designed to fit different levels of technical complexity and security requirements.

🔍 Prerequisites

You’ll need:

• Your AWS account logged in at https://console.aws.amazon.com
• Your S3 bucket name that has the assets to be uploaded
• Your AWS Account ID
• The Coactive IAM Role ARN:

1
arn:aws:iam::863104360228:role/coactive-external-connections

Method 1: IAM Role Assumption

Step 1: Go to the IAM Console

1. Go to the AWS Console: https://console.aws.amazon.com/iam
2. In the left sidebar, click “Policies”
3. In the top-right corner, click the “Create policy” button

🔐 Step 2: Create a Custom S3 Access Policy

You’ll now define a policy to allow Coactive to read your dataset.

1. In the “Create Policy” screen:

• Click the “JSON” tab at the top

2. Delete everything and paste in this content with your given S3 Bucket name

1
{
2
  "Version": "2012-10-17",
3
  "Statement": [
4
    {
5
      "Effect": "Allow",
6
      "Action": ["s3:Get*", "s3:List*"],
7
      "Resource": [
8
        "arn:aws:s3:::caitlin-demo-datasets",
9
        "arn:aws:s3:::caitlin-demo-datasets/10_images_10_videos/*"
10
      ]
11
    }
12
  ]
13
}

• arn:aws:s3:::caitlin-demo-datasets refers to the bucket itself — this allows Coactive to perform bucket-level operations such as listing objects (e.g., s3:ListBucket). This is required to view the contents or structure of the bucket during ingestion.
• arn:aws:s3:::caitlin-demo-datasets/10_images_10_videos/* refers specifically to all objects inside the 10_images_10_videos/ folder (also known as a prefix in S3). This grants Coactive permission to read the actual image and video files within that folder (e.g., s3:GetObject).

Together, these ensure that Coactive can:

1. See what objects exist in the dataset folder
2. Access and ingest the individual assets for processing

This approach adheres to AWS best practices by limiting access to only the necessary bucket and subfolder.

3. Click Next (bottom-right)

4. Add Policy Details

• Add a name: A clear and descriptive name indicating that this policy grants access to a specific S3 bucket in Caitlin’s AWS account, intended for Coactive’s ingestion process.

  caitlin_s3_access_policy

• Add a description (optional): Policy that allows Coactive to access images and videos in my S3 bucket.

• Click Create policy

• You’ll be returned to the “Policies” page. Your policy is now ready to attach to a role.

🛠️ Step 3: Create a New IAM Role

1. In the left sidebar, click “Roles”
2. In the top-right corner, click “Create role”

🧑‍🤝‍🧑 Step 4: Choose Trusted Entity

You’ll now define who can use (assume) this role.

1. Under “Trusted entity type”, select: AWS account

2. Under “An AWS account ID”, select Another AWS account and paste:

   863104360228

   (This is Coactive’s production AWS account)

3. Leave “Require external ID” unchecked

4. Click Next

   

📎 Step 5: Attach Your Custom S3 Access Policy

1. In the “Add permissions” screen:

   • Click the 🔄 refresh icon at the top-right of the list

   • In the search box, type:

     caitlin_s3_access_policy

2. Check the box next to that policy to select it

3. Click Next (bottom-right)

   

🏷 Step 6: Name and Create the Role

1. In the “Role name” field, enter:

   caitlin_external_ingest

2. (Optional) Description:

   IAM Role for Coactive to access my personal S3 demo dataset.

3. Click Create role

   

4. The role has now been created!

🔁 Step 7: Edit the Trust Relationship

Now you’ll tell AWS to trust only Coactive’s IAM role.

1. On the roles list page, click on:

   caitlin_external_ingest

2. Click the “Trust relationships” tab

3. Click “Edit trust policy”

4. Replace the existing text with this:

1
{
2
  "Version": "2012-10-17",
3
  "Statement": [
4
    {
5
      "Effect": "Allow",
6
      "Principal": {
7
        "AWS": "arn:aws:iam::863104360228:role/coactive-external-connections"
8
      },
9
      "Action": "sts:AssumeRole"
10
    }
11
  ]
12
}

Click “Update policy”

🔑 Step 8: Copy the Role ARN

You’ll need this to register the role with Coactive.

1. Still on the role details page, look for the “ARN” at the top
2. Copy it — it should look like this:

arn:aws:iam::<your-account-id>:role/caitlin_external_ingest

1. You’ve now created and configured everything in AWS, please use our ingestion APIs to ingest your assets.

Method 2: Connect Coactive Using a Bucket Resource Policy

This approach grants access directly to Coactive’s role via a bucket policy and no IAM role assumption is needed.

Step 1: Copy the Coactive Bucket Policy Template

Replace all instances of YOUR_BUCKET_NAME with your actual bucket name:

1
{
2
  "Version": "2012-10-17",
3
  "Statement": [
4
    {
5
      "Sid": "AllowCoactiveConnectionsListBucket",
6
      "Effect": "Allow",
7
      "Principal": {
8
        "AWS": "arn:aws:iam::863104360228:role/coactive-external-connections"
9
      },
10
      "Action": "s3:List*",
11
      "Resource": [
12
        "arn:aws:s3:::YOUR_BUCKET_NAME"
13
      ]
14
    },
15
    {
16
      "Sid": "AllowCoactiveConnectionsReadObjects",
17
      "Effect": "Allow",
18
      "Principal": {
19
        "AWS": "arn:aws:iam::863104360228:role/coactive-external-connections"
20
      },
21
      "Action": "s3:Get*",
22
      "Resource": [
23
        "arn:aws:s3:::YOUR_BUCKET_NAME/*"
24
      ]
25
    }
26
  ]
27
}

Step 2: Add the Policy to Your S3 Bucket

1. Go to the AWS Console: https://console.aws.amazon.com/s3
2. Click on your bucket: caitlin-demo-datasets
3. Go to the “Permissions” tab
4. Scroll down to “Bucket policy”
5. Click Edit
6. Paste the updated JSON into the editor
7. Click Save changes. Now Coactive’s role has read access to your bucket!

Step 3: Ingest Using the Fixed Connection Name "coactive"

You do not need to create a connection via API — just reference the special connection_name: "coactive" in your ingestion call.

Here’s a tailored example for you:

1
import httpx
2
3
http = httpx.Client()
4
5
response = http.post(
6
    "https://api.coactive.ai/api/v1/ingestion/assets",
7
    headers={
8
        "authorization": "Bearer ACCESS_TOKEN"
9
    },
10
    json={
11
        "dataset_id": "YOUR_DATASET_ID",  # Replace this with your real dataset ID
12
        "connection_name": "coactive",  # Key part here
13
        "assets": [
14
            {
15
                "source_path": "s3://caitlin-demo-datasets/10_images_10_videos/image1.jpg", # Replace this with your real source_path
16
                "metadata": {
17
                    "label": "example"
18
                }
19
            }
20
        ]
21
    }
22
)