Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW

Qlik Replicate Loading Best Practices

No ratings
cancel
Showing results for 
Search instead for 
Did you mean: 
Troy_Raney
Digital Support
Digital Support

Qlik Replicate Loading Best Practices

Last Update:

Feb 1, 2023 1:58:56 AM

Updated By:

Troy_Raney

Created date:

Jan 20, 2023 7:33:51 AM

This Techspert Talks session covers:

  • Full Load vs. Reload
  • Examining Task Options
  • Configuration Best Practices

Troy_Raney_0-1674217848224.gif

Environment:

  • Qlik Replicate

00:00 – Intro

00:58 - Full Load tasks

02:26 - Adjusting load priority

04:13 - Loading LOB fields

05:43 - Full Load Passthrough Filter

09:58 - Using Insert Date and Update Date

13:42 - Parallel Loading

15:28 - Setting up Notifications

18:23 - Q&A: Can Replicate do compressions?

18:45 - Q&A: What can affect performance?

20:03 - Q&A: How does metadata affect performance?

20:19 - Q&A: Is Replicate compatible with Oracle Data Bricks?

21:09 - Q&A: What if a task gets "Max Row Size Limit"?

21:58 - Q&A: Are Oracle OCI Plugable Databases supported?

22:39 - Q&A: What is the recommended version of Qlik Replicate?
23:59
- Q&A: How to limi or filter notifications?

24:58 - Q&A: How can you restart a stopped task?

25:49 - Q&A: Is Qlik Replicate free with Qlik Cloud?

26:11 - Q&A: How to change the Primary Key field?

27:21 - Q&A: How to investigate task latency?

28:15 - Q&A: What kind of hardware helps performance?

28:47 - Q&A: How to adjust segments to improve load performance?

Q&A:

Q: If I have 10 tables in a task, could I select only 1 table for a full load or does it have to be managed via a separate task?

A: Reload of a specific table is possible from the monitor screen. The whole task/all tables will reload at the task level.

Q: What is the best way to debug the CDC apply changes process?

A: Review the Troubleshooting Qlik Replicate Latency with Kelly Hobbs. She details this process.

Q: Do lookups affect latency of tables?

A: Yes, there is some overhead when the task issues the select statement on the source in order to get the data.

Q: If our target database is used by another development, will it be affected? Is there any possibility of data deadlock?

A: Yes, if the other application ever issues a lock on the target table rows that could cause an error for the Replicate task.

Q: Is there a way to setup a Notification that sends only a few e-mails when a Task has an Error or Task Stops because it can't connect to the Target and needs to stop? My experience has been that it fills my e-mail inbox with hundreds of pages.
A:  You can set up rules in your mail, or send notifications to the event viewer and have a program that scans the event viewer and sends emails based on the logic coded into the program.

Q: What happens in Replicate on a delete?

A: The delete will remove the record from the target. We do have a way to do a soft delete that will keep the record and mark it as deleted. Check out this article: Filter Deletes in Replicate

Q: The options available for tuning LOB handling when we use Log Stream as source as opposed to native application type source endpoint, are limited. So is there any way to explicitly pass the config options in task JSON configuration?

A: The LOB limitations on the LOG stream are by design and cannot be overridden.

 

Resources:

Customizing Data with Qlik Replicate

Adding Update Date field

Adding Table level Insert Date

Qlik Replicate Supported End Points

Troubleshooting Qlik Replicate Latency

An Introduction to Qlik Replicate Tasks: Full Load vs CDC

Techspert-Talks_Imagery_Community-590x310.jpg

Click here for video transcript

 

Version history
Last update:
‎2023-02-01 01:58 AM
Updated by: