topic Optimizing Data Reload Performance in Move to Qlik Cloud Analytics

Optimizing Data Reload Performance

theoat — Tue, 27 Jan 2026 10:45:26 GMT

Hello,

I need to reload a table daily that contains 60 million rows and 125 columns. I considered an incremental reload, but it doesn’t seem feasible since this is a snapshot of the previous day's stock levels.

Do you have any recommendations to speed up this reload on ? Previously, we performed this operation on , where the loading times were significantly shorter.

Thank you in advance for your help.

Enjoy your Qlik.

Kind regards,
Théo ATRAGIE.

Re: Optimizing Data Reload Performance

rubenmarin — Wed, 19 Mar 2025 08:25:53 GMT

Hi, do you really need all 125 columns? Retrieveing only the needed columns may reduce load times.

And the 60 millions rows is for the last day or it has historical data?

Also stocks with zero units exists on the table?

Re: Optimizing Data Reload Performance

marksouzacosta — Wed, 19 Mar 2025 12:38:53 GMT

Hi @theoat,

For large Data Sets I use a mix of Incremental Load and Data Partition. Since Incremental Load is not an option you could try Data Partition. The concept is simple. You pick a field (or fields) from your data set that does not change over time and that makes sense to group by. Often, I group my data by Year/Month. My QVDs/Parquets will look like:

...
stock_202412.qvd
stock_202501.qvd
stock_202502.qvd
stock_202503.qvd
...

You can even create a logic where you only process the current month plus the previous X months reducing drastically the extraction times.

Regards,

Mark Costa

Re: Optimizing Data Reload Performance

rwunderlich — Wed, 19 Mar 2025 15:20:28 GMT

To add to what @marksouzacosta suggested, here's an article on segmenting QVDs with some code samples.

https://qlikviewcookbook.com/2022/03/how-to-segment-qvd-files/

-Rob