Qlik Replicate | Using filter condition on table

[SreehariPV]

Hi team, 
I have a doubt regarding creating filter conditions for a column, in the attached image I have used two filter condition, I'm having a requirement were I want to use more number of filters that maybe in thousands(using python API QEM) , is it possible or can I know whats the maximum number of filters I can use.

Thankyou
Regards
Sreehari

[SreehariPV]

@Heinvandenheuvel Hi, we have considered this, but we have limited source access. So we didn't continue with this approach

[john_wang]

Hello @SreehariPV ,

Would you please elaborate the necessary of 20000 individual values rather than a range?

thanks,

John.

[DesmondWOO]

Hi @SreehariPV ,

Did you encounter hanging issue while defining a task or during task execution? What is your source database?

Regards,
Desmond

[SreehariPV]

The client had a requirement where they wanted to fetch only particular records( eg: 5 days of a month, it does'nt have to the same days in a week) based on two filter columns, one is a date field and the other is an id column. So adding them maually is an issue so I'm using QEM's python API

[SreehariPV]

The hanging issue was while opening the task and viewing the filters, the whole app gets stuck, I'm yet to run the task because I'm getting an error using the QEM's python API.
Source: Oracle
Target PostgreSQL

[Heinvandenheuvel]

Time to face the music. It's just not a workable design. Tell your customer.

If they have to add a filter ID, with the proposed design they have to engage a replicate person instead of an DB or business analyst person, then generate a new task, import it, stop and start the task, Too risky. 

Only half reasonable solution is a filter ID lookup table on source or target. That way the business can maintain it themselves and the task can keep on running. There will be a performance price of course.

Hein.

[john_wang]

Totally agree with @Heinvandenheuvel , utilizing VIEW(s) greatly enhances task design simplicity and performance.

John.

[SreehariPV]

While using run_task() what should I pass as payload, is it a json string or something else, what exactly should I pass to

AemRunTaskReq which is a payload parameter

[john_wang · Accepted solution]

Hello @SreehariPV ,

Would you please open a new article for the new issue? It's much easier for trace.

thanks,

John.