topic How much data can Qlik Sense handle? in Connectivity & Data Prep

How much data can Qlik Sense handle?

Mon, 05 Oct 2015 15:51:13 GMT

Hi,

I would like to use Qlik sense Server to analyse a big stack of data pushed everyday.

Every morning I have 2500 CSV files at my disposal, they weight up to 50 GB sometimes more..

I also want to compute year-to-date KPIs so this means a lot of data to digest!

How much time would it take to load 50gb and how would my dashboards perform after (let's say) 1 month of heavy loading?

Thanks for your help

Tom

Re: How much data can Qlik Sense handle?

hic — Mon, 05 Oct 2015 15:57:23 GMT

The answer is: A lot!

There are no hard limitations, except 2 Billion distinct values in each field.

The "soft" limitation is that the QIX engine needs to be able to load all data into physical RAM, which means that if you can equip your box with a large amount of RAM, you can load hundreds of millions of records.

But - 50gb of text files every day sounds to me as if you soon will reach a limit. So, you should consider ways to partition your data or pre-aggregate it. One app per week/month? An overview app? etc.

HIC