During Incremental Loads, Optimized Load Time Exploding

timothyj — Wed, 17 May 2023 18:37:48 GMT

I am loading incremental data from a data warehouse into QVDs. I use a SQL select query and then an optimized QVD load with Where Exists(Field_1). We have been doing this for months with no issue. We run our reloads triggered with JAMS, if that matters. On 4/29/2023 we updated QS, QV, and NPrinting. In Qlik Sense we went to Feb 2023.

On one load the initial incremental SQL load would take 6 or so minutes for 10 million rows. The Optimized QVD load would be about 270 million rows and load in 12 minutes. This was consistent for the time period I can see going back. Starting on 5/1, a few days after the upgrade the SQL load times have stayed the same. The optimized QVD loads can either be the same, suddenly take over 2 hours, or even jump up to 4 or 5 hours. I am checking the logs and the row count has increased slightly but only as expected. It is only the times that have increased. After a 5 hour load last night, I reran it through the browser and it only took 18 minutes, or what I expect.

Has anybody else had issues? Any ideas? This is new behavior.

Re: During Incremental Loads, Optimized Load Time Exploding

Albert_Candelario — Thu, 18 May 2023 08:29:31 GMT

Hello @timothyj ,

Thanks for posting.

The behavior you are reporting is very inconsistent.
What I would suggest you is getting the 2 scrips logs from the last days where it took 5h and 18min and review where the increase of time has been.

This will allow you to narrow down the possible issue if any.

Cheers,

Albert

topic Re: During Incremental Loads, Optimized Load Time Exploding in Management & Governance

During Incremental Loads, Optimized Load Time Exploding

Re: During Incremental Loads, Optimized Load Time Exploding