Using S3 Presigned URL for Data Upload with STORE command
Hello,
We are looking for advice with a system solution given the following preconditions/limitations.
Our organization strongly prefers to not use static credentials for AWS resources such as S3. This puts us in a situation where we're exploring alternate methods to connect to S3. As of now, S3 connectors only support static credentials.
Since Data Access Gateways can only be associated with a single Space/Tenant, we're architecting a system where QVD files can be uploaded to S3 as a mechanism to move data retrieved from a single Data Access Gateway to multiple tenants. Data must first be extracted to S3, as there is no download API available for QVD files.
The STORE statement can save the QVD files to a specified connection. We wanted to see if using a REST connector and pointing to a presigned URL for S3 upload would be viable. There are two problems to this approach:
REST connectors only support the GET and POST verbs. A standard presigned URL for uploading uses the PUT verb.
You can use POST method with presigned URL upload, but this requires a multi-form data submission. The REST Connectors POST method UI doesn't support any multi-form data submission capability.
Does anybody have any experience using REST connectors and S3 presigned URLs for uploading QVD files and can advise on what other options, if any, are available?
