topic Automating Data Load in Catalog and Lineage

Automating Data Load

kathysun — Tue, 07 Dec 2021 22:26:21 GMT

Hey,

I'm currently working on data loading in the Qlik Catalog. I'm curious if there's anyway to automate the process of loading data instead of manually refresh/load through the UI?

Best,

Kathy

Re: Automating Data Load

Hordy_Beauzile — Mon, 27 Jun 2022 18:37:57 GMT

Hi @kathysun,

As of now there is no way to automate the process of loading data within the UI. For that functionality you would have to post within our Community Ideation board here: https://community.qlik.com/t5/Ideation/ct-p/qlik-product-insight

Best,

Hordy

Re: Automating Data Load

brunnovb_in — Mon, 27 Jun 2022 21:26:09 GMT

Hello @kathysun,

You can automate using the API of Qlik Catalog. I've created a program using the call "/entity/v1/loadDataForEntities". It is possible create a script using bash (.sh) and schedule to execute on linux. This way I managed to automate this load process.

Regards,

Brunno.

Re: Automating Data Load

rick31 — Sun, 10 Sep 2023 12:37:08 GMT

It grabs files from a directory you specify and also based on the filename. So lets say you put files there to process with a specific name format, jitterbit picks it up, processes it and can edit the file name (like add 'processed [date]' at the end of the file and save the processed file in a different location.