How can my flink job sink to an S3 bucket while Ververica Platform's universal blob storage uses another S3 bucket with different credentials?

By default, VVP uses one set of S3 credentials (the platform/instance credentials) for everything including checkpoints, BlobStorage, and any S3 sink. This article shows how to give a specific sink bucket its own credentials using Hadoop S3A per-bucket configuration, while the checkpoint/BlobStorage bucket keeps using the platform credentials untouched.

1. Prepare the sink IAM credentials
   Create (or reuse) an IAM user whose policy grants access to the sink bucket. Note that both bucket-level and object-level actions are required similar to the following example:
   

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "s3:ListBucket",
           "s3:GetBucketLocation"
         ],
         "Resource": "arn:aws:s3:::<YOUR-VVP-BUCKET>"
       },
       {
         "Effect": "Allow",
         "Action": [
           "s3:GetObject",
           "s3:PutObject",
           "s3:DeleteObject",
           "s3:AbortMultipartUpload",
           "s3:ListMultipartUploadParts"
         ],
         "Resource": "arn:aws:s3:::<YOUR-VVP-BUCKET>/*"
       }
     ]
   }

2. Create a set of credentials

   Generate an access key for the user (IAM → Users → [user] → Security credentials → Create access key) and copy both the Access key ID and Secret access key from the confirmation screen.

3. Inject the credentials in your flinkConfigurations

   Add the credentials in your deployment flinkConfigurations as follows:

   fs.s3a.bucket.<YOUR-VVP-BUCKET>.access.key: '<YOUR-ACCESS-KEY>'
   fs.s3a.bucket.<YOUR-VVP-BUCKET>.secret.key: '<YOUR-SECRET-KEY>'

   Make sure you quote both the access and secret keys in single quotes.