topic Re: Performance of load script with combined fields in App Development

Performance of load script with combined fields

jjustingkm — Sat, 26 Jul 2025 19:20:46 GMT

I have a table with around 60 million records and 25 fields, and some fields are combined in the table level as below. Does it make it any difference in loading performance combining those columns or brining them independently? And also does it make more time during indexing after load?

Company ID Name Region ID Name

123 ABC International 004 North

Re: Performance of load script with combined fields

anat — Sun, 27 Jul 2025 06:35:14 GMT

You can load fields independently and you have to split if you have date time fields

Re: Performance of load script with combined fields

marcus_sommer — Mon, 28 Jul 2025 11:47:16 GMT

You may not mandatory need the name- respectively descriptive fields within the facts else they might be kept within appropriate dimension-tables.

Re: Performance of load script with combined fields

Oleg_Troyansky — Mon, 28 Jul 2025 12:17:50 GMT

Hi there,

I'd say, from my experience combining fields in this manner (code + description) should not terribly impact the load performance. To the contrary, assuming that the code and the description are related 1:1, the combo field will have the same cardinality as the two separate fields. Hence, as a result, some of the index space should be saved by using one field instead of two fields. And, while I don't know the exact mechanics of indexing after the load, I wouldn't expect that process to be majorly impacted.

You may experience a minor slowdown from the act of concatenating two strings, but that should not be way too bad, IMHO.

The only possible scenario where this could cause substantial performance issue, is this - if you are loading 60 million rows from a QVD file, you should definitely make sure that the load is optimized. Concatenating fields during a QVD load makes the load non-optimized. Now, that would cause a noticeable slow-down.

Allow me to invite you to the Masters Summit for Qlik in Hamburg, or to my Qlik Expert Class in Vienna, where I teach advanced development techniques, including Performance Optimization. With the high data volumes that you are working with, it helps to know advanced Qlik methodologies that I teach.

Cheers,

Oleg Troyansky