Yes, Loops are really slow for what you are trying to do.
I would chose another approch, creating a load of your cars table with a integrated WHILE loop in the LOAD statement to generate a record per bit of the mask. Then MAP the bits:
CarTypeBitMask: MAPPING LOAD * INLINE [ _CarTypeMaskVal, _CarTypeMaskText 1,Truck 2,Car 4,Mini Van 8,Cross-Over 16,2-wheel 32,4-wheel 64,AWD ]; //And the list of cars read in from a data source: (example made up, I'm not really dealing with cars) [Cars]: LOAD * INLINE [ %CarID,CarTypeMask 1,18 2,33 3,24 ]; // //The idea is to generate a new table pulling out the car type combinations: // //CarTypes: //%CarID,CarType //1,Car //1,2-Wheel //2,Truck //2,4-Wheel //3,Cross-Over //3,2-Wheel // // //Here's the script I run, but with thousands of rows it is way too slow: // // CREATE CarTypes 1-N table // RESULT: LOAD * ,APPLYMAP('_CarTypeBitMask', Split, 'Oops, missing CarTypeBitMap entry') as CarType WHERE Split; LOAD %CarID, CarTypeMask bitand pow(2,iterno()-1) as Split Resident Cars WHILE iterno() < 7;
If this runs better, could you post the improvement?
Thank you very much!
This is a very slick piece of code - I continue to forget that you can iterate during the load operation.
This is SIGNIFICANTLY FASTER!
I had a table of 30,000 entries and 16 possible bit masks.
The original script took 2 hours to run. With your solution it was down to 30 seconds !!