Qlik Replicate advance run datetime with db2 as source

[lguevara]

Hi.

I have a task with a DB2 iSeries source and an S3 target, configured for full load and store changes. The task started its first execution 4 weeks ago. Now, I want to stop the task and only capture changes from 2 weeks ago. However, when I set the date 2 weeks ago in the advanced run options under 'date and time', nothing loads into the full load table. How can I apply the load from that specific point in time, considering the source is DB2 iSeries and the target is S3?

Thanks

[john_wang · Accepted solution]

Hello @lguevara ,

Qlik Replicate tasks can include both a Full Load phase and a CDC (Change Data Capture) phase (if both are enabled in the task settings).

Initial run behavior

1. The Full Load phase completes first and loads the initial dataset to the target.
2. The task then automatically transitions to the CDC phase and continues applying ongoing changes.

These two phases are executed seamlessly.

Behavior when restarting from a specific timestamp

1. The Full Load phase is effectively skipped (no initial data is reloaded).
2. The CDC phase resumes from the specified timestamp and reprocesses changes.

Note: Since some of these changes may have already been applied previously, this can lead to:

• duplicate rows
• primary key violations
• or other inconsistencies (depending on task settings)

Suggested approach for your scenario

1. Enable Full Load and disable CDC
2. Apply a Fullload Passthru Filter to load only a subset of the initial data into a “full load” target table
3. Run the task and wait for the Full Load to complete
4. Then enable CDC and restart the task from a specific timestamp (e.g., two weeks ago)

The changes will be captured from that point onward and applied to your “changes” table.

This approach helps separate initial data loading from historical change capture while maintaining better control over data needs.

Hope this helps.

John.

[lguevara]

Hi @john_wang ,

When the task is executed using the date-time option—for example, starting from four days ago the latency increases to that point in time. How long will it take to process all that backlogged data, and what factors does the processing time depend on?

thanks.

[john_wang]

Hello @lguevara ,

It depends on many factors, for example:

1. The volume of data in the transaction logs
2. Network bandwidth
3. The IBM DB2 for i CPU capacity
4. The target S3 write throughput/performance
5. Overall system workload and concurrency etc

Regards,

John.