Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us in Toronto Sept 9th for Qlik's AI Reality Tour! Register Now
cancel
Showing results for 
Search instead for 
Did you mean: 
jjustingkm
Creator II
Creator II

Performance of load script with combined fields

I have a table with around 60 million records and 25 fields, and some fields are combined in the table level as below. Does it make it any difference in loading performance combining those columns or brining them independently? And also does it make more time during indexing after load?

Company ID  Name                        Region ID Name

123 ABC International                    004  North

Labels (1)
3 Replies
anat
Master
Master

You can load fields independently and you have to split if you have date time fields

marcus_sommer

You may not mandatory need the name- respectively descriptive fields within the facts else they might be kept within appropriate dimension-tables.

Oleg_Troyansky
Partner Ambassador/MVP
Partner Ambassador/MVP

Hi there,

I'd say, from my experience combining fields in this manner (code + description) should not terribly impact the load performance. To the contrary, assuming that the code and the description are related 1:1, the combo field will have the same cardinality as the two separate fields. Hence, as a result, some of the index space should be saved by using one field instead of two fields. And, while I don't know the exact mechanics of indexing after the load, I wouldn't expect that process to be majorly impacted.

You may experience a minor slowdown from the act of concatenating two strings, but that should not be way too bad, IMHO.

The only possible scenario where this could cause substantial performance issue, is this - if you are loading 60 million rows from a QVD file, you should definitely make sure that the load is optimized. Concatenating fields during a QVD load makes the load non-optimized. Now, that would cause a noticeable slow-down. 

Allow me to invite you to the Masters Summit for Qlik in Hamburg, or to my Qlik Expert Class in Vienna, where I teach advanced development techniques, including Performance Optimization. With the high data volumes that you are working with, it helps to know advanced Qlik methodologies that I teach.

Cheers,

Oleg Troyansky