Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
AlexOmetis
Partner Ambassador
Partner Ambassador

Amazon RDS for the repository database with Shared Persistence? (& related questions)

Having watched ltu's informative webinar on Shared Persistence, we're interested in how this could benefit our customers who run or are considering running Qlik Sense on AWS.

My questions are, roughly in order of priority...

  1. Assuming you're splitting the repo db from the central node, can you use an Amazon RDS postgres database as the repository database rather than having another actual instance spun up to run Postgres?
  2. Assuming you're taking the root folder off the central node, can you use an Amazon S3 for this rather than having another actual instance spun up to store this on?
  3. Are there any recommendations around configuring and optimising Shared Persistence in Amazon AWS?

Cheers

Alex

Qlik Partner Ambassador 2024
1 Solution

Accepted Solutions
Levi_Turner
Employee
Employee

  1. RDS for PostgreSQL: Yes
  2. S3 for files: No. S3 does not have the IOPS that Sense will need.

If you're looking down the path of optimizing every step of the Sense deployment, then upgrading the disks attached to the AWS instance is in play (e.g. provisioned IOPS). In reality, this is more of an edge case in the sense of being required for high concurrency. For most deployments, extra time spent optimizing the data model of the app to require less computational resources by the Engine is a larger value add overall.

View solution in original post

13 Replies
Anonymous
Not applicable

Yes for 1 and 2 with additional efforts and configurations. I dont have much insights for item 3.

Levi_Turner
Employee
Employee

  1. RDS for PostgreSQL: Yes
  2. S3 for files: No. S3 does not have the IOPS that Sense will need.

If you're looking down the path of optimizing every step of the Sense deployment, then upgrading the disks attached to the AWS instance is in play (e.g. provisioned IOPS). In reality, this is more of an edge case in the sense of being required for high concurrency. For most deployments, extra time spent optimizing the data model of the app to require less computational resources by the Engine is a larger value add overall.

AlexOmetis
Partner Ambassador
Partner Ambassador
Author

Thanks for the response ltu - very helpful.

For #2 I should probably have asked about other storage options too - could EFS perform well enough? Or another storage-only offering? Or would you recommend an instance with disk attached that shares with the other nodes?

Qlik Partner Ambassador 2024
AlexOmetis
Partner Ambassador
Partner Ambassador
Author

Have done some research and looks like EFS would be ideal for Qlik Sense on AWS. Shared access, good I/O, multiple zone access, presents as a mounted drive etc...

Instances for nodes but RDS for repository and EFS for storage seems like the way forwards.

Qlik Partner Ambassador 2024
Not applicable

Hi Alex,

Could you share some instructions of setting up RDS and EFS?

AlexOmetis
Partner Ambassador
Partner Ambassador
Author

Haven't actually set it up yet, so can't at the moment. Will try and keep good notes if/when I do implement and provide a write-up. Don't know if @ltu or anyone has notes already. There's a deployment guide available but the current version doesn't reference Shared Persistence nor RDS (other than as a data source) nor EFS from my quick scan of it - not sure if that'll be of help for you.

Qlik Partner Ambassador 2024
AlexOmetis
Partner Ambassador
Partner Ambassador
Author

We're about to commence doing this - with RDS but not EFS at this stage. If anyone ( ltu?) has done it recently and has any tips, that'd be much appreciated!

Qlik Partner Ambassador 2024
neo_lee
Partner - Creator
Partner - Creator

Basically, setup an RDS Postgres, make sure the port required is open in your AWS security group. You can then connect to the RDS during the QS installation take place. Insert the RDS DNS name, the username you specify for your RDS, and the rest will be worked like magic.

However, I haven't got a clue how to restore a QS node that is based on AWS RDS. Dumping SQL is easy, but surely there is a config variable file to point it to the new RDS DNS like a replica to test QS upcoming upgrades etc.

Levi_Turner
Employee
Employee

alex.walker : I have little light on the EFS aspect of things. RDS is simple enough. neo.lee‌ pointed out that the AWS security groups will need to be sorted out.

For EFS does it live on a domain or is it domain-less? If it lives on your domain, then that's straight-forward. I suspect, though, it's domain-less. In which case you can leverage the Windows Vault to cache a set of credentials for a remote server (saved as the svc account) which should work.

I'll see if I can find anything a consultant has put together.