topic Re: Redshift S3 Staging Credentials in Qlik Replicate

Redshift S3 Staging Credentials

simonB2020 — Tue, 24 May 2022 17:39:28 GMT

If I choose "IAM Roles for EC2", then I am also asked for "IAM Role ARN"
I am assuming that Replicate uses the former to write to S3, and the latter as part of Redshift's COPY command ?

However,
If I choose "Key Pair", then I am not asked for an "IAM Role ARN".
So does Replicate then use the Key pair for both writing & COPY ?

Any reason for the different approach between the two approaches of single/dual credentials ?

Many Thanks

Re: Redshift S3 Staging Credentials

Steve_Nguyen — Tue, 24 May 2022 18:07:51 GMT

@simonB2020

1. for both S3 staging IAM or Key , Replicate will write to S3 and then copy to Redshift.

2. The option was offer because some customer want to use IAM and some want to use key pair, performance is the same.

Note from the user guide on IAM :

IAM Roles for EC2

Choose this method if the machine on which Qlik Replicate is installed is configured to authenticate itself using an IAM role.

For more information about this access option, see:

http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html

https://help.qlik.com/en-US/replicate/May2022/Content/Replicate/Main/Amazon%20Redshift/set_up_redshift_as_target.htm

Re: Redshift S3 Staging Credentials

mattdevdba — Tue, 24 May 2022 18:14:50 GMT

These are two different ways to authenticate.

IAM Roles for EC2 when used generate temporary security credentials (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY i.e. the key pair) for your session.

Key pair is a static AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

Best practice on AWS would be to use IAM roles. If you are following the AWS well architected framework or have an architecture review through the AWS Partner Network (APN) there is now criteria that says static AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY should not be used in this scenario.

It's also easier to manage as normally you have to have a process to rotate the credentials every 30-90 days if you are working in any kind of regulated environment. This management overhead just goes away with IAM Roles as they generate new short lived credentials on the fly.