topic Re: Load QVDs in an optimize way in QlikView

Load QVDs in an optimize way

jmvilaplanap — Wed, 25 Nov 2020 16:16:04 GMT

Hi all,

I need to load a huge QVD in qlikview (this QVD is from a previous stage of my dashboard) the "problem" is because I need to do a few transformations to this data but I want a good performace.

If I do the transformations in the LOAD statement I lost the optimize load, but is the same using a precedent load. (this load takes around four minutes).

If I load the QVD and later I do the transformations in a resident table, it will be faster? I don´t think so.

Then, What´s the best way to do it?

Thanks at all.

Re: Load QVDs in an optimize way

YoussefBelloum — Thu, 30 Nov 2017 09:42:01 GMT

when i read your question i directly remember a document from last week.

it doesn't completely answer your question, but take a look. it is about loading performance

It's not just your old fashioned INLINE statement

Re: Load QVDs in an optimize way

PrashantSangle — Thu, 30 Nov 2017 09:44:35 GMT

Before storing into QVD can you do all transformation???

If yes then try that approach.

Regards,

Re: Load QVDs in an optimize way

jmvilaplanap — Thu, 30 Nov 2017 10:05:05 GMT

Hi,

It´s a very good idea to use it to do something as a "where" but my transformations are different, I need to use Ifs or change formats (maybe Ifs could be done with INLINE tables and ApplyMap, but I think this is not an optimize way)

Thanks a lot

Re: Load QVDs in an optimize way

marcus_sommer — Thu, 30 Nov 2017 13:15:47 GMT

Like Prashant hinted if possible do the (essential) transformations one step before. You could also try it with loading the qvd optimized and performing the transformations on a resident-load but I doubt that there is much difference between both methods.

But you could try to implement an incremental load-approach. It's not only useful for loading rawdata else it could be very powerful within a second/third data-layer on qvd's - the logic itself is always the same. More to this and the necessary where exists() clause could you find here: Advanced topics for creating a qlik datamodel.

- Marcus