topic Re: Going for shared persistence? in Management & Governance

Going for shared persistence?

jaisoni_trp — Thu, 05 Jan 2017 22:31:38 GMT

For various reasons/bugs Qlik is recommending to upgrade Sense environment to Shared Persistence. There are significant architecture changes which comes with 3.1 and wanted to have customers opinion if you are ok upgrading the environment to 3.1 from 3.0. What are the challenges you have come across, any insight you could share.

Re: Going for shared persistence?

simondachstr — Wed, 18 Jan 2017 09:19:40 GMT

Qlik recommends shared persistence because their initial approach to a synchronized multi-node environment, called replication synchronization, simply does not work. It works, until it fails, and then it's almost impossible to bring up the services without calling in Qlik's support engineers (fyi, Andrew Delaney from Qlik is your repository ninja if you have issues).

Hence, shared persistence was developed last year, which synchronizes the multi-node environment differently. Instead of replicating all apps and repository content across all nodes (rep sync) it hosts everything on a central node and distributes the apps into the memory of each slave node. This way, still the whole infrastructure is still in sync in the eyes of the user. But on the back-end, if everything is only kept in RAM on the rim nodes, it implies a HOT-DR setup is not feasible (Unless you have a smart DFS replication setup for your database server). If the main server (where all the apps are stored) fails over, no reloads are possible and the apps are only available in RAM on the rest of the infrastructure (e.g. a server restart would wipe the in-memory loaded app out).

Replication synchronization on the other hand ensures all nodes (rim & central nodes) have the apps and repository replicated locally (can be customized using sync rules), meaning that while in sync, each node can also fully operate on its own, effectively making it an ideal hot DR solution.

My personal recommendation is the following:

- Shared persistence (Shared synchronization) is not mature enough as it has only been released in a hasty approach to remedy existing rep sync issues and has, in my eyes, not been proven to work reliably yet (it has not been unreliable either).

- Replication synchronization is very powerful in concept, but unfortunately not bulletproof in QS.

-> If you plan to have a small environment with not too many apps (<50) and 2-3 nodes, go with rep-sync. If on the other hand you want to be able to scale horizontally (3+ servers) with multiple apps and/or the use of frequent on-demand app generation, please look to go with shared persistence. You will need to setup a separate DR environment though, making it a bit more costly.

Re: Going for shared persistence?

simondachstr — Thu, 23 Feb 2017 16:29:27 GMT

So, apparently with 4.0 Qlik Sense will deprecate replication synchronization.

Re: Going for shared persistence?

jaisoni_trp — Thu, 23 Feb 2017 18:11:23 GMT

We have a large deployment with multi nodes. Migrated to Shared persistence on 3.1.4. Unfortunately started seeing lock erros in apps resulting in publishing failures.

Re: Going for shared persistence?

korsikov — Fri, 03 Mar 2017 08:06:22 GMT

thanks for your explanation.

have information whether in version 4.0 synchronized? my guess is that standalone version, in fact synchronized and shared persistense with native migration.

Re: Going for shared persistence?

Anonymous — Fri, 03 Mar 2017 22:53:57 GMT

We have been trying to make QS work for us since Version 1.1. Stability due to synchronization related issues have been biggest challenge. If your deployment is small, best to go for single node installation. If it is VM which is replicated to DR site, you are all covered.

If deployment is large, then moving to shared persistence is definitely recommended. New version 3.2 is looking comparatively better. 3.1 has serious repository query issues which cause extremely poor performance and unexpected behavior.

Re: Going for shared persistence?

mountaindude — Sat, 04 Mar 2017 21:44:19 GMT

I very much agree with the recommendation to go with shared persistence (SP) for anything larger than 2-3 nodes and/or 100+ apps.

We have been running SP for quite a few months now (with significant numbers of apps users and apps), and while not 100% perfect, I would say it's 90% there. Performance can always be improved, there are features in the QMC you wish were there etc - but all in all SP is solid. Performance improvements in 3.2 will only make it better.

Re: Going for shared persistence?

alexbjorlig — Mon, 20 Mar 2017 09:03:06 GMT

Do you have a link to a Qlik article supporting this statement?

Re: Going for shared persistence?

simondachstr — Mon, 20 Mar 2017 09:09:32 GMT

Which statement of all?

Re: Going for shared persistence?

simondachstr — Mon, 20 Mar 2017 09:11:45 GMT

All multi-node environments are synchronized - they just use a different approach.

With 4.0, shared persistence will be the only way to deploy a multi-node QS architecture going forward.

Re: Going for shared persistence?

jaisoni_trp — Mon, 20 Mar 2017 09:14:07 GMT

In case you are talking about decommissioning of synchronized persistence.

Qlik Sense 3.2 Service Release 1 now available

Please pay particular attention to pages 18 and 19 in the release notes which deal with the current technical requirements and deprecation of Synchronized Persistence planned for the next Sense feature release.

Re: Going for shared persistence?

alexbjorlig — Mon, 20 Mar 2017 09:34:48 GMT

Accodring to the article Jai Soni linked - is it not synchronized persistance that will be depreceated and not shared persistence? Or did I miss something?

Re: Going for shared persistence?

simondachstr — Mon, 20 Mar 2017 09:41:11 GMT

Sorry - that was a mixup! I have corrected my comment. Thanks for pointing out

Re: Going for shared persistence?

alexbjorlig — Mon, 20 Mar 2017 09:46:56 GMT

No problem. I was just reading this post and was super confused Btw - do you know if it is possible to upgrade from replication to shared persistence?

Re: Going for shared persistence?

Anonymous — Mon, 20 Mar 2017 14:34:13 GMT

For large deployment best to wait until 3.2.3. We upgraded to 3.2.0 because of 3.1.x issues however ran into serious problem with scheduler. Master scheduler will timeout while starting if you have more than couple of thousand reload task. It will come up after several hours of retry if lucky. We have identified several other issues as well in this version. As per support most of them will be fixed in 3.2.3 release.

Re: Going for shared persistence?

crusader_ — Wed, 29 Mar 2017 12:19:01 GMT

Hi Alex.

Yes, it's possible. I'm not sure whether it could be done by clicking on the button, but if you copy DB and restore on new environment, fix repository service config and move Apps, AppContent, Extensions etc files to share - that will work.

Re: Going for shared persistence?

Anonymous — Tue, 16 Jan 2018 18:12:43 GMT

The best plan of action is to upgrade to Qlik Sense, Shared Persistence, February 2018 release. All aforementioned issues and concerns have been addressed.