Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
We recently upgraded to November 2023 (Qlik Sense Enterprise) and noticed by accident that an ApplyMap works despite forgetting to add MAPPING to the previous table load. I can't find this being added as a feature - perhaps it's a happy accident?
This would save us considerable time when loading large QVDs. Currently, we load a QVD optimized then RESIDENT for a mapping load, taking almost as long as an unoptimized load. If we could load optimized into a mapping load and do an apply map, that would save significant time and memory resources. Of course, we have to drop the table later. I don't want to adjust our code to use this new (non?)feature if there's potential for it to break later.
If it's not a feature, but rather a happy accident, can we request it be added as a feature??
This is news to me. I can recreate it just as you've said. although I don't see any changes in the doc. In testing I found that the table referenced in the ApplyMap() must still have only two columns or an error is thrown.
I also note that :
Map x using y;
still requires that the "y" table be created with the Mapping prefix.
I don't quite understand your issue with optimized QVD load. As far as I can see you still can't use ApplyMap() in an optimized load. Can you post an example?
-Rob
http://www.easyqlik.com
http://masterssummit.com
http://qlikviewcookbook.com
My issue with the optimized load is for bringing in the data for the MAPPING LOAD not the ApplyMap.
The conventional syntax is:
// Proper syntax:
map_desc:
MAPPING LOAD
ITEM_ID,
DESC
RESIDENT description;
if we want to load a large QVD (e.g. 60M item metadata file) just to do a mapping load for a description:
// this load is NOT optimized.
map_desc:
MAPPING LOAD
ITEM_ID,
DESC
FROM [item.qvd] (qvd);
Our workaround:
// Optimized QVD load
temp_item:
LOAD
ITEM_ID,
DESC
FROM [item.qvd] (qvd);
// Memory load from temp to mapping (takes time)
map_desc:
MAPPING LOAD
ITEM_ID,
DESC
RESIDENT temp_item;
DROP TABLE temp_item;
Currently experiencing in Nov 2023 (hope this becomes a documented feature):
// optimized. requires DROP TABLE after ApplyMap
map_desc:
LOAD
ITEM_ID,
DESC
FROM [item.qvd] (qvd);
Hi @bsanders21,
You should not use large data sets to populate a Mapping Table. The performance is terrible.
Do a LEFT JOIN instead. Your code should load way faster.
Regards,
Mark Costa
Just chiming in to say that this does seem to be new to November 2023 - I have a May 2023 environment and that one still requires the Mapping keyword.
Potentially a useful shortcut here if it's intentional and permanent.
Understood, we use both, but unfortunately we utilize Qlik for significant ETL at times and a mapping load can be more efficient than a Left Join with a large volume of transactions in our FACT data (easily 500M+), especially when we have multiple ApplyMaps. We often test both ways for optimization depending on our project.
FYI I have tested on Qlik Cloud and I got the same behavior of QS November 2023
@Patric_Nordstrom Any chance you'd be aware of this? I think you're the go-to person on what's new in each version (at least based on the technical previews). Sorry if that's not the case!