Can you perform parallel reload operations with a split

MichelLalancette — Fri, 15 Mar 2024 15:05:33 GMT

I'm trying to build automations.

Our most common use-case is similar to this:

Extract from database
Transform data, making usable by many
If transform succeeded, start 2 datamart reloads, for 2 applications, both independent in case one fails
When Datamart 1 is done, start App 1 (front-end)
When Datamart 2 is done, start App 2 (front-end)

Basic schema of the workflow:

I have not found anything that really allows multiple parallel reloads without using possibly loops or something similar.

I thought about using 1 automation for the Extract+Transform then creating 2 separate automations for both Datamart/App combo and calling them in parallel but, I am wondering if it's doable in a single automation without being overly complex.

Regards,

Michel

Re: Can you perform parallel reload operations with a split

cotiso_hanganu — Fri, 15 Mar 2024 15:27:54 GMT

Not the same, but might bring some inspiration:

I remember that in Qlikview times we had to do a 8 lanes parallel reload tasks.
In order to do that we had 8 different slave QVW files (fully identical, just renamed Slave1,...8.QVW ) with reloads launched simultaneously from the QMC. Each was identifying the execution data subset to work with via a small semaphor file (QVD) and marking the current job as "assumed and started" so that no other one to start working on the same job. After ending the respective job, saving in the semaphor status file as solved for that job and moving further if anyone left still not executed.

Thinking these days to do a similar approach for importing via SOAP web services in SAP transactions from other system. (No Qlik Automation, just Qlik Sense scripting with POST via a REST API connection)

Sempre fi,
Cotiso

topic Re: Can you perform parallel reload operations with a split in Qlik Automate

Can you perform parallel reload operations with a split

Re: Can you perform parallel reload operations with a split