topic Re: Qlik replication - Filter condition in Qlik Replicate

Qlik replication - Filter condition

trksrini — Thu, 02 May 2024 17:41:59 GMT

My Task without filter condition has already completed full load, now I want to add a new filter condition for CDC records to exclude some records based on source column value.

I tried with adding below filter condition and record selection condition but both scenarios I am getting weird results:

Col Name is Cloud_migrt_ind.

CLOUD_MIGRT_IND

STRING(1)

Regular

Exclude

'Y'
$CLOUD_MIGRT_IND != 'Y'

Can you suggest the best way to add filter condition to exclude record based on column value ?

Thanks,

Srinivas

Re: Qlik replication - Filter condition

Heinvandenheuvel — Thu, 02 May 2024 19:17:13 GMT

Define weird.

Re: Qlik replication - Filter condition

trksrini — Thu, 02 May 2024 19:23:48 GMT

Sometime the records are skipped, working as expected when it met with filter condition(exclude condition) but not all the time..

Re: Qlik replication - Filter condition

trksrini — Thu, 02 May 2024 19:24:07 GMT

Sometime the records are skipped, working as expected when it met with filter condition(exclude condition) but not all the time..

Re: Qlik replication - Filter condition

Heinvandenheuvel — Thu, 02 May 2024 20:19:06 GMT

" but not all the time.. "

Then you must very carefully, very detailed look at the filter field.

Is is a 1 byte character column? Could it be an lower-case vs uppercase issue? UTF-8 issue?

What is the source? I recommend a DUMP (for Oracle, or CONVERT to HEX) for the column for the source row(s) that passed the filter and you believe should not have.

>> My Task without filter condition has already completed full load,

Could that have loaded the 'weird' rows? Did you restart with reload after the filter was added as recommended or did you resume by timestamp "tables already loaded". The latter would have left behind the rows which were already filtered. .

Hein.

Re: Qlik replication - Filter condition

john_wang — Fri, 03 May 2024 01:29:53 GMT

Besides @Heinvandenheuvel , I ever summarized the 3 filter conditions in article the 'filter' function is undertook by 3 different areas. We may check the source DB values as Hein mentioned or execute the same query manually in source DB to understand the reason.

Hope this helps.

John.

Re: Qlik replication - Filter condition

trksrini — Mon, 06 May 2024 13:21:01 GMT

Hello Hein,

Source DB is DB2 oz/OS, And target is Aurora pgsql. My requirement is filter some rows during CDC after full load is completed as we designed to update these records on Target directly and don't want to be synced from Source.

Yes, chose to restart the task using advance run option by timestamp as this option allows not to initiate full load.. Resume option is forcing to reload even full load option is turned off in task setting.

Thanks,

Srinivas

Re: Qlik replication - Filter condition

john_wang — Tue, 07 May 2024 02:29:58 GMT

Hello @trksrini ,

Thanks for the update. Regarding

>> Resume option is forcing to reload even full load option is turned off in task setting

Would you please elaborate the scenario? It's not reasonable. I'd like suggest opening support ticket with task Diag Packages and detailed steps, Our support team will be more than happy to assist you to investigate the behavior deeper.

Regards,

John.