[Qlik Sense : Load Editor] Joining a table to a partition of another table

[Ylian]

Hello everybody,

I have several tables containing a lot of data (like 25M rows per table) on which I am doing joins. To optimize the calculation time, I would like to make these joins only on certain parts of my tables.

For example: let two tables A and B both include a "month" field. I would like to make a join between these two tables knowing that the rows with "January" from table A will give interesting results than with the rows with "January" in table B. Currently, I am doing a simple join between A and B but this gives a high calculation time because for each row of table A we browse the entire table B.

In my real case, there are many tables with several million rows each time.

I thought of using "WHERE" but for the question of the optimization of the calculation time: is it applied before or after the calculation of the join?

Also, is there any solution for partitioning tables in Qlik that could solve this problem?

Thank you for your future response!

ylian

[JHuis]

There are many different options for this.

You could use a concatenate to concatenate tables. 

You could use a where exist to only get to relevant data from a second table. 

But maybe if you give us a sample of your data i could help you out.

[Ylian]

The objective is to reduce calculation times by avoiding traversing the entire table when it is known that the interesting data is for a given month.

[JHuis]

Incremental load no option?

[Ylian]

No it's not an option because, on the one hand, the data is cleaned regularly and we don't know what is deleted or not and, on the other hand, all our data is already loaded upstream in the load editor because we already use it for other things.