Skip to main content

Connectivity & Data Prep

Discussion board where members can learn more about Qlik Sense Data Connectivity.

Announcements
QlikWorld 2023, a live, in-person thrill ride. Save $300 before February 6: REGISTER NOW!
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

How much data can Qlik Sense handle?

Hi,

I would like to use Qlik sense Server to analyse a big stack of data pushed everyday.

Every morning I have 2500 CSV files at my disposal, they weight up to 50 GB sometimes more..

I also want to compute year-to-date KPIs so this means a lot of data to digest!

How much time would it take to load 50gb and how would my dashboards perform after (let's say) 1 month of heavy loading?

Thanks for your help

Tom

1 Reply
hic
Employee
Employee

The answer is: A lot!

There are no hard limitations, except 2 Billion distinct values in each field.

The "soft" limitation is that the QIX engine needs to be able to load all data into physical RAM, which means that if you can equip your box with a large amount of RAM, you can load hundreds of millions of records.

But - 50gb of text files every day sounds to me as if you soon will reach a limit. So, you should consider ways to partition your data or pre-aggregate it. One app per week/month? An overview app? etc.

HIC