topic Re: Can Attunity configure delay processing when doing data extraction？ in Qlik Replicate

Can Attunity configure delay processing when doing data extraction？

FWD_Qlik_Support — Wed, 22 Dec 2021 10:54:41 GMT

Hi Attunity teams,

I have a question to ask you，Can Attunity configure delay processing when doing data extraction? Or when extracting data, can you write sql statements for conditional filtering and extraction?

The scenario is like this: when doing the lottery, there is a logic that needs to use two tables, denoted as Table A and Table B, respectively. The data of Table A depends on Table B, that is, the data of Table B must precede Table A. Is extracted, but the actual business is that the data of table A and table B are generated at the same time. I need to delay the extraction of table A (that is, after the extraction of table B is completed, table A is extracted), or I control through sql, the query is less than a certain The data at the time point is extracted.

Can you check if there is any support for me? Thanks!

Re: Can Attunity configure delay processing when doing data extraction？

Maria_Halley — Wed, 22 Dec 2021 12:29:53 GMT

@ FWD_Qlik_Support

I will move this to the QDI section, so you reach the right audience

Re: Can Attunity configure delay processing when doing data extraction？

Heinvandenheuvel — Wed, 22 Dec 2021 15:01:05 GMT

Any Replicate based solution for this business problem will be clumsy and unreliable.

imho you should really look at a solution inside the application, maybe control table indicating whether a table can be used or not, maybe a flag row or column in the tables, maybe a RENAME.

How could Replicate ever find out, reliably, that table B is fully loaded and table A can be 'activated?

Maybe A can be a view on the real A which included a select/filter cause which only passed rows when something special is visible in table B or another helper table?

If you want to solve this in Replicate, then maybe you must just make a dedicated task for table A which is stopped when fresh B contents is generated and started (by timestamp?) when B is fully available.

Hein

Re: Can Attunity configure delay processing when doing data extraction？

Dana_Baldwin — Wed, 22 Dec 2021 23:53:47 GMT

Hi @FWD_Qlik_Support

As Hein mentions above, there isn't a way to guarantee this behavior. One related feature that is available in Replicate is you can specify a priority for tables to load in on the Designer tab, Selected tables section at the bottom right. You can make Table B highest priority and Table A lowest priority, and see if that helps. If the number of tables in the task is relatively low, or the number of rows in table B is relatively high, it might not provide enough of a delay.

Dana

Re: Can Attunity configure delay processing when doing data extraction？

Heinvandenheuvel — Thu, 23 Dec 2021 04:18:23 GMT

Hi Dana. Good suggestion.

I assumed the table reloading was an ongoing thing.

But if a full-load tasks is used, priority could be used for a solid solution providing the max streams is set to 1 (default is 5). With sufficient other tables with sufficient load times I suppose you could risk a larger number of concurrent streams.

Hein