Qlik Community

QlikView Layout & Visualizations

Discussion Board for collaboration on QlikView Layout & Visualizations.

sandipsatbhai
Contributor II

What will be the best approach to handle the huge amount of data(50 millions of rows)

How does QlikView handle large amounts of data? is there any way to design dashboard with huge amount of data?

7 Replies

Re: What will be the best approach to handle the huge amount of data(50 millions of rows)

With a well-build datamodel could a dasboard run very smoothly with 50 M records even with quite small hardware-ressources and you could heavily slow down an application with a few k records if you makes (many) things wrong.

There is no best approach to build a datamodel and an application - it will be always depend on the kind of data, the number of user and the requirements of the business which data should be calculated and displayed in which way. Therefore you need to be a lot more specific with your question.

- Marcus

Re: What will be the best approach to handle the huge amount of data(50 millions of rows)

see the below link in which there is some resources for the best practices

Large amount of data (120 million rows) ... anyone got experience?

MVP
MVP

Re: What will be the best approach to handle the huge amount of data(50 millions of rows)

Hi,

Check few points below

- Do you really need all the data?  Check with user how many past years data they want? Last 5, 10 or N.  This will reduce the file size, Records count, if file size is less QV performs well.

- Load only the required columns.

- Truncate Timestamp if you are not using it for analysis

This are the main points to consider while creating a data model.

Regards,

Jagan.

sandipsatbhai
Contributor II

Re: What will be the best approach to handle the huge amount of data(50 millions of rows)

Thank you all for your awesome reply...

MVP
MVP

Re: What will be the best approach to handle the huge amount of data(50 millions of rows)

Adding another classic reference to all what has been already said above:

*** 6 Weeks in to QV Development, 30 Million Records QV Document and Help Needed!!! ****

sandipsatbhai
Contributor II

Re: What will be the best approach to handle the huge amount of data(50 millions of rows)

Very good thread... Thanks a lot.. swuehl ,

luciancotea
Valued Contributor

Re: What will be the best approach to handle the huge amount of data(50 millions of rows)

Always remember, it's not the number of rows that matters, it's the number of distinct values in each column.

Community Browser