topic Re: Avoiding loading data already loaded in QlikView

Avoiding loading data already loaded

Zaga_69 — Sat, 16 Nov 2024 04:59:02 GMT

Dear all,

I am extracting data from a View from SAP HANA that has around 10 millions rows.

My script in Qlik Sense is organized as follows:

Main

Extract

Transform

Hierarchy

Facts

Dimensions

In the "Transform" part I made some changes on the data coming from the "Extract" part like changing field names or converting date string fields into date field and so on.

However, every time I need to change sth in the "Transform" part I'm loading the data again and it is a waste of time.

Do you know how make changes in the "Transform" but avoiding load the data again?

Best regards,

Edi

P.S: Once I extract the data I stored it into qvd tables.

Re: Avoiding loading data already loaded

Somasundaram — Wed, 12 Dec 2018 13:18:23 GMT

Hi Ediftayo,

Extract-- In extract part you need to do format the values

Transform-- All the transforms done here only.

Use incremental load to avoid load existing data.

Re: Avoiding loading data already loaded

Zaga_69 — Wed, 12 Dec 2018 13:20:36 GMT

Can you please provide a code example to do that 🙂 ?

Thanks

Re: Avoiding loading data already loaded

Somasundaram — Wed, 12 Dec 2018 13:29:21 GMT

Please check below links,

https://community.qlik.com/t5/QlikView-Documents/Incremental-load-for-beginner-with-example/ta-p/1486886

https://www.analyticsvidhya.com/blog/2014/09/qlikview-incremental-load/

and try to implement with your data.

Re: Avoiding loading data already loaded

Zaga_69 — Wed, 12 Dec 2018 13:33:55 GMT

Thank you 🙂

Much appreciated.

Best regards,

Edi