topic Re: Is a Mapping indicator still required for Mapping Loads? in App Development

Is a Mapping indicator still required for Mapping Loads? (works without in QS Nov'23)

bsanders21 — Mon, 19 Aug 2024 18:46:00 GMT

We recently upgraded to November 2023 (Qlik Sense Enterprise) and noticed by accident that an ApplyMap works despite forgetting to add MAPPING to the previous table load. I can't find this being added as a feature - perhaps it's a happy accident?

This would save us considerable time when loading large QVDs. Currently, we load a QVD optimized then RESIDENT for a mapping load, taking almost as long as an unoptimized load. If we could load optimized into a mapping load and do an apply map, that would save significant time and memory resources. Of course, we have to drop the table later. I don't want to adjust our code to use this new (non?)feature if there's potential for it to break later.

If it's not a feature, but rather a happy accident, can we request it be added as a feature??

Re: Is a Mapping indicator still required for Mapping Loads?

rwunderlich — Sat, 17 Aug 2024 02:53:54 GMT

This is news to me. I can recreate it just as you've said. although I don't see any changes in the doc. In testing I found that the table referenced in the ApplyMap() must still have only two columns or an error is thrown.

I also note that :

Map x using y;

still requires that the "y" table be created with the Mapping prefix.

I don't quite understand your issue with optimized QVD load. As far as I can see you still can't use ApplyMap() in an optimized load. Can you post an example?

-Rob
http://www.easyqlik.com
http://masterssummit.com
http://qlikviewcookbook.com

Re: Is a Mapping indicator still required for Mapping Loads?

bsanders21 — Mon, 19 Aug 2024 13:47:40 GMT

My issue with the optimized load is for bringing in the data for the MAPPING LOAD not the ApplyMap.

The conventional syntax is:

// Proper syntax: map_desc: MAPPING LOAD ITEM_ID, DESC RESIDENT description;

if we want to load a large QVD (e.g. 60M item metadata file) just to do a mapping load for a description:

// this load is NOT optimized. map_desc: MAPPING LOAD ITEM_ID, DESC FROM [item.qvd] (qvd);

Our workaround:

// Optimized QVD load temp_item: LOAD ITEM_ID, DESC FROM [item.qvd] (qvd); // Memory load from temp to mapping (takes time) map_desc: MAPPING LOAD ITEM_ID, DESC RESIDENT temp_item; DROP TABLE temp_item;

Currently experiencing in Nov 2023 (hope this becomes a documented feature):

// optimized. requires DROP TABLE after ApplyMap map_desc: LOAD ITEM_ID, DESC FROM [item.qvd] (qvd);

Re: Is a Mapping indicator still required for Mapping Loads?

marksouzacosta — Mon, 19 Aug 2024 13:49:35 GMT

Hi @bsanders21,

You should not use large data sets to populate a Mapping Table. The performance is terrible.
Do a LEFT JOIN instead. Your code should load way faster.

Regards,

Mark Costa

Re: Is a Mapping indicator still required for Mapping Loads?

Or — Mon, 19 Aug 2024 13:50:12 GMT

Just chiming in to say that this does seem to be new to November 2023 - I have a May 2023 environment and that one still requires the Mapping keyword.

Potentially a useful shortcut here if it's intentional and permanent.

Re: Is a Mapping indicator still required for Mapping Loads?

bsanders21 — Mon, 19 Aug 2024 13:53:36 GMT

Understood, we use both, but unfortunately we utilize Qlik for significant ETL at times and a mapping load can be more efficient than a Left Join with a large volume of transactions in our FACT data (easily 500M+), especially when we have multiple ApplyMaps. We often test both ways for optimization depending on our project.

Re: Is a Mapping indicator still required for Mapping Loads?

marksouzacosta — Mon, 19 Aug 2024 14:24:04 GMT

FYI I have tested on Qlik Cloud and I got the same behavior of QS November 2023

Re: Is a Mapping indicator still required for Mapping Loads?

Or — Mon, 19 Aug 2024 14:42:26 GMT

@Patric_Nordstrom Any chance you'd be aware of this? I think you're the go-to person on what's new in each version (at least based on the technical previews). Sorry if that's not the case!