System requirements for large files

PaoloGuerra — Thu, 24 Feb 2022 15:23:57 GMT

We have QlikSense Desktop for Analytics on csv files.

We have problems with errors and freezing in the indexing phase.

What is the System requirements to index csv files for a total of 15/20 gb?

This is the hardware currently available.

We also have the ability to install virtual machines.

Can we help me please?

Re: System requirements for large files

Daniele_Purrone — Fri, 25 Feb 2022 08:52:06 GMT

Hi @PaoloGuerra ,
the first thing to consider is that the Qlik Sense engine is an in-memory tool, so you'll need enough physical memory to handle the data. Virtual memory is usually not an option.

Then it depends on what the data is about, and what you'll do with it.
The Engine stores unique values only once, so, after a while, the curve for RAM usage flattens.
https://community.qlik.com/t5/Qlik-Sense-Documents/The-data-scalability-of-Qlik-s-in-memory-associative-data-model/ta-p/1479181

You can probably test this by adding "FIRST" before the load statement, followed by a limited number of records.
https://help.qlik.com/en-US/sense/February2022/Subsystems/Hub/Content/Sense_Hub/Scripting/ScriptPrefixes/First.htm

Test it with a certain number of records, check the resource usage, then do subsequent tests with a larger number of records until you get an idea of what the curve will look like.