Is an optimized LOAD with a RESIDENT afterwards faster?

[datanibbler]

Hi,

I have read some articles on optimized and unoptimized qvd_loading, but now I have a specific question:

=> I am just looking at an app that, for some reason, loads about 200mio records from a qvd in an unoptimized LOAD - that is apparently because there is one keyfield being constructed from two separate fields in that LOAD, otherwise it is a plain LOAD that could be optimized.

Is it worthwhile doing an optimized LOAD on this and then doing a RESIDENT LOAD where the keyfield can be constructed? Would that still be faster?

Thanks a lot!

Best regards,

DataNibbler

[datanibbler]

Hi Pankaj,

right now, QlikView is our ETL tool. There are tendencies to use something else, but actually we are trying to more or less Keep things as they are because this is what we do and this is what the Company Needs us for 😉 There are also tendencies to move the "real" development work out of the Company to externals, like everybody else seems to be handling this - so we are struggling to Keep the justification of our being here ...

That said, yes, I am usually trying to move complex calculations away from the frontend and further "upstream" wherever possible. That's where my next question Comes in:

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

I have looked more closely into the generation of this calculated field (which is the sole reason why I Need one RESIDENT LOAD in the app I was actually looking at):

The app before is one Monitoring that specific EXTRACT and it works like a historization: The existing qvd is loaded, six or seven blocks of code append 20 or 30 records each and then it's saved again. That means I won't gain anything by pushing the RESIDENT LOAD creating that field there because it would be just as huge, but I have to do it individually in every one of those small chunks - that will, in sum, be a lot faster than doing it on the huge table in one go - since I would be repeating a lot of what I have done before every time ...

Well, that will have to wait until there's time to spend on testing that.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Then I can have another look at that app which I was primarily looking at: Right now, I have one optimized qvd-LOAD at that point, then a RESIDENT to calculate that one field - and then one more to append it to an empty table which is initiated by the following code

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Fehlerdaten:
NoConcatenate load
0 as Fehlercode
AutoGenerate (0);

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

and then all that is appended - is that really necessary? I don't understand what is the reason to do that.

(if that turned out to not be necessary I could do away with one of the RESIDENTs)

Can you tell me which (if any) added_value there is in initiating a table and then appending to that empty

construct?

Thanks a lot!

Best regards,

DataNibbler