Qlik Community

QlikView App Development

Discussion Board for collaboration related to QlikView App Development.

amien
Valued Contributor

How do you manage complex ETL hierarchy tasks?

I have a dummy QMC task. This is kick off a range of QVW reloads. After this is done, other tasks are being fired (using the on multi-able complete event tasks for example). I try to manage kicking off several tasks because if i would push all the script into 1 qvw file, its much slower. By splitting it i want to source more CPU to do the tasks.

what i'm quite missing in QMC is some kind of intelligence (like SSIS and other ETL tools have). that kicks off a new tasks, even if an other tasks is not finished yet. The tool just know, ok, i now have enough rows for now, to start the next task, and meanwhile i wait for the next batch of records comes in. I'm now try to put specifiek timers on the tasks and work with dependancies. But in the end this will get un-maintainable.

I'm also missing a 'trigger if unsuccessful reload' (there isn't one right?). when something gets broke into the reload chain, it should start all over. I now created some kind of green/red toggle. It prevent the head-dummy to kick off the chain again, while the chain is still running.

I wonder how you manage this all, when a reload needs to be done several times an day/hour

2 Replies
Highlighted
Partner
Partner

Re: How do you manage complex ETL hierarchy tasks?

Are you using QlikView Publisher?

In publisher this should be possible, or else you could create an EDX task

amien
Valued Contributor

Re: How do you manage complex ETL hierarchy tasks?

Yes, we have publisher.

Where are the options as described as above in QMC?