Amazon RDS for the repository database with Shared Persistence? (& related questions)

[AlexOmetis]

Having watched ltu's informative webinar on Shared Persistence, we're interested in how this could benefit our customers who run or are considering running Qlik Sense on AWS.

My questions are, roughly in order of priority...

1. Assuming you're splitting the repo db from the central node, can you use an Amazon RDS postgres database as the repository database rather than having another actual instance spun up to run Postgres?
2. Assuming you're taking the root folder off the central node, can you use an Amazon S3 for this rather than having another actual instance spun up to store this on?
3. Are there any recommendations around configuring and optimising Shared Persistence in Amazon AWS?

Cheers

Alex

[Anonymous]

Yes for 1 and 2 with additional efforts and configurations. I dont have much insights for item 3.

[Levi_Turner · Accepted solution]

1. RDS for PostgreSQL: Yes
2. S3 for files: No. S3 does not have the IOPS that Sense will need.

If you're looking down the path of optimizing every step of the Sense deployment, then upgrading the disks attached to the AWS instance is in play (e.g. provisioned IOPS). In reality, this is more of an edge case in the sense of being required for high concurrency. For most deployments, extra time spent optimizing the data model of the app to require less computational resources by the Engine is a larger value add overall.

[AlexOmetis]

Thanks for the response ltu - very helpful.

For #2 I should probably have asked about other storage options too - could EFS perform well enough? Or another storage-only offering? Or would you recommend an instance with disk attached that shares with the other nodes?

[AlexOmetis]

Have done some research and looks like EFS would be ideal for Qlik Sense on AWS. Shared access, good I/O, multiple zone access, presents as a mounted drive etc...

Instances for nodes but RDS for repository and EFS for storage seems like the way forwards.

[#6]

Hi Alex,

Could you share some instructions of setting up RDS and EFS?

[AlexOmetis]

Haven't actually set it up yet, so can't at the moment. Will try and keep good notes if/when I do implement and provide a write-up. Don't know if @ltu or anyone has notes already. There's a deployment guide available but the current version doesn't reference Shared Persistence nor RDS (other than as a data source) nor EFS from my quick scan of it - not sure if that'll be of help for you.

[AlexOmetis]

We're about to commence doing this - with RDS but not EFS at this stage. If anyone ( ltu?) has done it recently and has any tips, that'd be much appreciated!

[neo_lee]

Basically, setup an RDS Postgres, make sure the port required is open in your AWS security group. You can then connect to the RDS during the QS installation take place. Insert the RDS DNS name, the username you specify for your RDS, and the rest will be worked like magic.

However, I haven't got a clue how to restore a QS node that is based on AWS RDS. Dumping SQL is easy, but surely there is a config variable file to point it to the new RDS DNS like a replica to test QS upcoming upgrades etc.

[Levi_Turner]

alex.walker : I have little light on the EFS aspect of things. RDS is simple enough. neo.lee‌ pointed out that the AWS security groups will need to be sorted out.

For EFS does it live on a domain or is it domain-less? If it lives on your domain, then that's straight-forward. I suspect, though, it's domain-less. In which case you can leverage the Windows Vault to cache a set of credentials for a remote server (saved as the svc account) which should work.

I'll see if I can find anything a consultant has put together.