topic Re: Data amount spread in tables in QlikView

Data amount spread in tables

Thu, 07 Jan 2016 07:01:28 GMT

Hi,

Is there a difference for the QlikView engine if I have 10 tables, each with 10,000,000 rows and 40 columns, or if I have 2 tables with 50,000,000 rows and 40 columns?

(an app contains 10-20 calculated objects based on this tables)

I want to understand it for 2 scenarios and how they would affect the performance:

1) When the tables are connected Via a Key

2) When they are not

Thanks,

Boris

Re: Data amount spread in tables

manojkulkarni — Thu, 07 Jan 2016 07:07:24 GMT

In both the cases, 10tables is much faster compare to 2 tables of bigger size. For any selection, rather going through 50M records, filtering 10M record is much faster. But if the filter selections are coming from different tables (multiple tables)then we have problem with 10M record tables.

Re: Data amount spread in tables

Thu, 07 Jan 2016 07:16:14 GMT

In the second part of your reply, what do you mean filter selections that come from different tables?

For example, I have 10 queries that that run in Hadoop Hive and create 10 files (tables).

each file has the same data structure.

The only field which is different is a Date field which sometimes contains a day date and some times a month date.

In this case, should I concatenate all files into one table or leave it in 10 tables in memory with same data structure?

Regards,

Re: Data amount spread in tables

Thu, 07 Jan 2016 07:22:37 GMT

Concatenate the tables. QlikView's indexing function stores pointers for each distinct value and it's relationship in the data model. This will be always be faster

Re: Data amount spread in tables

Thu, 07 Jan 2016 07:49:27 GMT

Thanks Byron!