topic Re: Qlik Sense error loading "big" table in App Development

Qlik Sense error loading "big" table

Sebastian_P — Wed, 09 Aug 2023 13:09:22 GMT

Hi ,

[QS2023 May 14.129.6 WIN 2012 R2 Standard, Intel(R) Xeon(R) E5-2620 v2 @2.10Ghz , RAM-16 GB]

I have a trouble while loading 25mln table (30 columns) from qvd (extraction of 1 table only).

Different cases happen:

- cannot save the app

- engine stops

- error 129

- table is loaded partly e.g. 6mln

- works (rarely)

Memory usage does not exceed 80% during load (Task monitor).

Problem accured 2 months after upgrade to May 2023.

It looks like shortage of RAM but there was no problem for few years, so far.

Any ideas?

Thanks.

Re: Qlik Sense error loading "big" table

Or — Wed, 09 Aug 2023 13:24:46 GMT

Some thoughts:

* If possible, try increasing memory availability to see if that solves the problem or not

* Check that there is enough disk space on any relevant drives (the server machine's Windows/Qlik Sense drives and the one holding the app)

* If this is not the only thing being loaded, make sure that there isn't an issue with an incorrect join statement to another table or something similar

Re: Qlik Sense error loading "big" table

Chirantha — Wed, 09 Aug 2023 13:25:09 GMT

As for the engine stopping could be due to high memory consumption. When you stop the Qlik Sense Engine Service, the engine.exe will keep running until it has released the memory, which can take a long time

Qlik Sense Engine Service stops but engine.exe sti... - Qlik Community - 1717251

Re: Qlik Sense error loading "big" table

marcus_sommer — Wed, 09 Aug 2023 13:25:11 GMT

The qvd-load is optimized or with any transformation? Does anything changed within the script before the qvd-load is executed? From where is the qvd loaded - any network or a local storage and has this changed recently respectively anything else within the environment?

Re: Qlik Sense error loading "big" table

Sebastian_P — Wed, 09 Aug 2023 14:50:31 GMT

Thanks for replies,

There is only one simple load, no transformation or where clause.
disk is local
enough disk space
no info about changes in environmet - probably nothing changed

I plan to update to patch 3 and probably add some ram.

Re: Qlik Sense error loading "big" table

marcus_sommer — Wed, 09 Aug 2023 14:59:35 GMT

I would assume by your mentioned dataset and machine a load-time between 3 and 10 minutes by an optimized qvd-load (depending on the kind of data and the disk-speed) and it should progress quite constantly if you are looking on the progress-window.

If the load itself runs but afterwards the creation of the data-model hangs it might be caused by any synthetic keys and/or circular references in regard to other loaded tables.

Re: Qlik Sense error loading "big" table

Sebastian_P — Thu, 17 Aug 2023 18:23:06 GMT

Additional ram - nothing changed.

Patch 3 - did the job!