topic Re: Source database restores -- how does Qlik replicate handle in Qlik Replicate

Source database restores -- how does Qlik replicate handle

zacker20 — Mon, 19 Jun 2023 15:51:07 GMT

We are running SQL Server (MS-CDC) and our company does a database restore from Production to our TEST environment every week or so. We are pulling the TEST databases as a source to our lower data warehouse environments using LogStream.

When the restores happen, Replicate breaks the change processing and cannot pick up the CDC tables. Is there anyway around RELOADING all the tasks on the server? This takes an incredibly long time and is an inconvenience for our developers.

Re: Source database restores -- how does Qlik replicate handle

john_wang — Tue, 20 Jun 2023 01:05:13 GMT

Hello @zacker20 ,

Thanks for contacting Qlik Support.

I did not understand the question well... Do you mean you have many tasks on a Replicate Server, after the TEST database (acts as source database for these tasks) restores, you want to RELOAD (perform both Full Load + CDC) startup on all these tasks, is that right?

If yes, then we may do that by script file. Let us know the details.

Regards,

John.

Re: Source database restores -- how does Qlik replicate handle

zacker20 — Tue, 20 Jun 2023 01:14:59 GMT

Hey John,
No I want to see if we can avoid doing a full reload. The full reloads take a long time - leaving our lower environments down until they complete.

We have two tasks with over 10 billion records total between the two of them — reloading these tasks can take days.

Can Qlik pick up after a restore of a db?

Re: Source database restores -- how does Qlik replicate handle

john_wang — Tue, 20 Jun 2023 01:40:01 GMT

Hello @zacker20 ,

Thanks for the clarification.

From my understanding, you need startup tasks by CDC only run options without Full Load - because we assume the history rows were replicated from source to target already, we just want to catch up the CDC part after each time restore. The best option is resume task by timestamp

Hope this helps.

Regards,

John.

Re: Source database restores -- how does Qlik replicate handle

zacker20 — Tue, 20 Jun 2023 01:48:59 GMT

No the target is not restored. Only the source db is restored.

Re: Source database restores -- how does Qlik replicate handle

john_wang — Tue, 20 Jun 2023 02:12:32 GMT

Hi @zacker20 ,

Yes, we are talking about the source db restores. Let me know if you have any concerns.

Regards,

John.

Re: Source database restores -- how does Qlik replicate handle

zacker20 — Tue, 20 Jun 2023 02:55:59 GMT

So you’re saying the load from date/time would work? How would that work with the db being restored with a db from production? Records that may have been changed in the past in the test environment would still remain in our Qlik environment right? Records could be missing?

Re: Source database restores -- how does Qlik replicate handle

zacker20 — Tue, 20 Jun 2023 12:18:20 GMT

When we tried to do this load from date/time - we got a ton of apply exception errors as time went on. UPDATE statements would have 0 rows affected