Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
I have a table with around 60 million records and 25 fields, and some fields are combined in the table level as below. Does it make it any difference in loading performance combining those columns or brining them independently? And also does it make more time during indexing after load?
Company ID Name Region ID Name
123 ABC International 004 North
You can load fields independently and you have to split if you have date time fields
You may not mandatory need the name- respectively descriptive fields within the facts else they might be kept within appropriate dimension-tables.
Hi there,
I'd say, from my experience combining fields in this manner (code + description) should not terribly impact the load performance. To the contrary, assuming that the code and the description are related 1:1, the combo field will have the same cardinality as the two separate fields. Hence, as a result, some of the index space should be saved by using one field instead of two fields. And, while I don't know the exact mechanics of indexing after the load, I wouldn't expect that process to be majorly impacted.
You may experience a minor slowdown from the act of concatenating two strings, but that should not be way too bad, IMHO.
The only possible scenario where this could cause substantial performance issue, is this - if you are loading 60 million rows from a QVD file, you should definitely make sure that the load is optimized. Concatenating fields during a QVD load makes the load non-optimized. Now, that would cause a noticeable slow-down.
Allow me to invite you to the Masters Summit for Qlik in Hamburg, or to my Qlik Expert Class in Vienna, where I teach advanced development techniques, including Performance Optimization. With the high data volumes that you are working with, it helps to know advanced Qlik methodologies that I teach.
Cheers,
Oleg Troyansky