Connectivity & Data Prep

[#1]

Hi,

I would like to use Qlik sense Server to analyse a big stack of data pushed everyday.

Every morning I have 2500 CSV files at my disposal, they weight up to 50 GB sometimes more..

I also want to compute year-to-date KPIs so this means a lot of data to digest!

How much time would it take to load 50gb and how would my dashboards perform after (let's say) 1 month of heavy loading?

Thanks for your help

Tom

[hic]

The answer is: A lot!

There are no hard limitations, except 2 Billion distinct values in each field.

The "soft" limitation is that the QIX engine needs to be able to load all data into physical RAM, which means that if you can equip your box with a large amount of RAM, you can load hundreds of millions of records.

But - 50gb of text files every day sounds to me as if you soon will reach a limit. So, you should consider ways to partition your data or pre-aggregate it. One app per week/month? An overview app? etc.

HIC