Qlikview maximum scale

[#1]

Hello all,

I would like to know if someone has some knowledge regarding the max scale of data that QV can work with and give seconds of response time on it's dashboards.

Currently we are working with data sets of more than 1 billion records and we can't get any normal functionality in our dashboards.

We are working on user level aggregations (we handle 100 million user bases and more with history data).

Will appreciate any insights on the data scale matter.

Regards,

Kacevich Boris

[tresesco]

Check the attached doc; should be helpful.

[#3]

the benchmarks are to low.

if 10 mil is low and 50 mil is medium then what is a large data set? 100 mil? does that mean the QV can't work with more than 200 mil lets say?

[avinashelite]

check this document

[tresesco]

No. QlikView doesn't have such limitation. As long as your hardware (RAM, CPUs..) permits/cope up with, number of records could not hit the limit. Only limitation is with number of distinct values which is 2 billions (approx).

[SreeniJD]

Hi Boris,

To be practical there is no "Unlimited" in this technical world at one point application will behave very strange and will not give values.. sometimes if you have huge data, data might not be loaded properly or after a freasing point you can even experience the "negative values" to be loaded in your tables...

So, follow best practices like compressing data, storing data in QVDs, 4 tier architecture and ofcourse with good hardware, we can overcome some size limitations

HTH

Sreeni

[#7]

is it true for responsiveness in the UI?

also, we are working with a 256GB servers with more than 16 cores and we can't handle 500 million + data sets.

what kind of server do I need to handle this scales?

[tresesco · Accepted solution]

QlikView demands a lot of memory(RAM) and CPU. And when I say 'a lot', it means really a lot. 256GB RAM would seem to be a very big RAM with other applications. But when it comes to in-memory technologies, it might be just a good size. I have seen much more RAM and CPUs to be used for qlikview. There are some standard calculations available to decide about your server sizing. Check here: Re: Qlikview server sizing

[#9]

In other words, if you use the NASA computer then the sky is the limit...

in my case, when I work with 100 million subscribers and about 3 billion rows of daily aggregations then I can't use QV even with strong hardware such as a 256 GB RAM and 64 Cores machine.

Have to say that we do load the data first to QVD's and then load them in memory for further calculations.

Still doesn't help, the UI is barely loading and most of the time unresponsive.