Is it possible to use Filter Conditions and Record Selection Conditions on the same column / table / task?
Hello,
I'm wondering if it's possible to use Filter Conditions and Record Selection Conditions on the same column / table / task?
What I'm hoping to do is use the Filter Conditions to first limit what Replicate extracts, then use the Record Selection Conditions to do a more complex or dynamic filtering before writing to the target.
My source has billions of rows from the last two decades, but only want to dynamically replicate the last two years of data.
Yes — you can use Filter Conditions and Record Selection together in Qlik Replicate, but you’ll want to make sure they don’t overlap or conflict in logic. For example, if your filter condition excludes certain records and your selection tries to include them (or vice-versa), the filters will take precedence and those records will be excluded anyway.
When both are used, the replication engine applies the filters first to limit incoming data, and then within those remaining records your record‐selection settings determine which get processed further.
Please note the differences between 'Filter Conditions' and 'Record Selection Conditions.' For example, during a full load, in the 'Filter Conditions,' the filter is applied within the select statement. In contrast, with 'Record Selection Conditions,' Qlik Replicate retrieves all records and performs the filtering internally.
Please also check the limitations related to using filters.
Regards, Desmond
Help users find answers! Do not forget to mark a solution that worked for you! If already marked, give it a thumbs up!
Filters let you include or exclude records from a replication task based on the value(s) of the source table columns, thereby allowing you to replicate only the specific data that you need.
As per your requirement, I would suggest you use the full load passthrough method to replicate the last two years of data. The full load pass thru filter is more efficient since it filters directly on the source. For this to work, you need to use the exact syntax of the source database. It can only be applied to full load tasks, not change data capture mode.
Please check the articles below for more information.
