Qlik Community

QlikView App Development

Discussion Board for collaboration related to QlikView App Development.

Highlighted
Not applicable

Data amount spread in tables

Hi,

Is there a difference for the QlikView engine if I have 10 tables, each with 10,000,000 rows and 40 columns, or if I have 2 tables with 50,000,000 rows and 40 columns? 

(an app contains 10-20 calculated objects based on this tables)

I want to understand it for 2 scenarios and how they would affect the performance:

1) When the tables are connected Via a Key

2) When they are not

Thanks,

Boris

4 Replies
Partner
Partner

Re: Data amount spread in tables

In both the cases, 10tables is much faster compare to 2 tables of bigger size. For any selection, rather going through 50M records, filtering 10M record is much faster. But if the filter selections are coming from different tables  (multiple tables)then we have problem with 10M record tables.

Not applicable

Re: Data amount spread in tables

In the second part of your reply, what do you mean filter selections that come from different tables?

For example, I have 10 queries that that run in Hadoop Hive and create 10 files (tables).

each file has the same data structure.

The only field which is different is a Date field which sometimes contains a day date and some times a month date.

In this case, should I concatenate all files into one table or leave it in 10 tables in memory with same data structure?

Regards,

Not applicable

Re: Data amount spread in tables

Concatenate the tables. QlikView's indexing function stores pointers for each distinct value and it's relationship in the data model. This will be always be faster

Not applicable

Re: Data amount spread in tables

Thanks Byron!