Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 
al3x
Contributor II
Contributor II

parallel load - table segmentation method

Hello Qlik Community

I want to use prallel load for a table. The option "Use Partitions" is blurred. 

Using the option "Use Data Ranges" causes the problem, that we don't have a equaly distributed column on the source database. 

Here is a made up example. Using the year column isn't very efficient for a data range, because some ranges will get bigger than others. 

al3x_0-1705487083540.png

It would be nice to have something like:

Segment 1: PRIMARY_KEY mod 4 = 0
Segment 2: PRIMARY_KEY mod 4 = 1
Segment 3: PRIMARY_KEY mod 4 = 2
Segment 4: PRIMARY_KEY mod 4 = 3

Can you propose some solution or workaround?

Regards
al3x

Labels (3)
1 Solution

Accepted Solutions
Dana_Baldwin
Support
Support

Hi @al3x 

This is not a currently available option. I encourage you to request support for this directly from our Product Management team here: https://community.qlik.com/t5/Ideas/idb-p/qlik-ideas

For help implementing this with the current capabilities, you might consider contacting our Professional Services team. Please check with your Account Manager on this option.

Thanks,

Dana

View solution in original post

3 Replies
DesmondWOO
Support
Support

Hi @al3x ,

The option "Use Partitions" is disabled, likely because your table is not configured as a partitioned table.

In regards to the data skew issue, it would be advisable to consult with your DBA. Resolving this issue requires a thorough understanding of your table structure and data distribution. For instance, if your table has a primary key, such as an invoice number with incrementally increasing values, you could use this column to distribute data evenly, as opposed to using the year.

Regards,
Desmond

Help users find answers! Do not forget to mark a solution that worked for you! If already marked, give it a thumbs up!
al3x
Contributor II
Contributor II
Author

Hello Desmond,

Thank you for your informative reply. That would be an option for me if otherwise not possible.

I am looking for a solution that doesn't involve changing the condition from time to time. While using a "primary key, such as an invoice number with incrementally increasing values," I would need to analyze the table structure and add or change the pipeline definition from time to time.

I'm looking for a more elegant and future-proof solution, e.g., modulo operations.

Is there something like that, or is Qlik planning to have it in a future release?

Regards, Alex

Dana_Baldwin
Support
Support

Hi @al3x 

This is not a currently available option. I encourage you to request support for this directly from our Product Management team here: https://community.qlik.com/t5/Ideas/idb-p/qlik-ideas

For help implementing this with the current capabilities, you might consider contacting our Professional Services team. Please check with your Account Manager on this option.

Thanks,

Dana