topic Re: How does filter on a source table work internally? in Qlik Replicate

How does filter on a source table work internally?

eksmirnova — Wed, 03 May 2023 21:43:09 GMT

Hi everyone,

I have a Full load Qlik task with a filter condition:

source - Oracle,

target - Snowflake

How does filter work internally? Does QLIK pull entire table from Oracle and only then apply the filter condition?

The table which I am pulling is huge. It has 100,000,000 rows but it has monthly partitions.

My filter is : date($MY_REPORT_DT) = date(current_date, '-1 day')

Re: How does filter on a source table work internally?

Alan_Wang — Wed, 03 May 2023 22:00:23 GMT

Hi @eksmirnova

Qlik Replicate does pull the entire table before filtering out the records. You can consider filtering on the database level with the Full Load Passthru filter, but it does open another connection to the source database.

More information can be found in this article.

https://community.qlik.com/t5/Official-Support-Articles/Filter-for-last-90-days-of-data-in-Qlik-Replicate/ta-p/1880761

Re: How does filter on a source table work internally?

eksmirnova — Wed, 03 May 2023 22:07:09 GMT

@Alan_Wang Thank you!