Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi All,
We are unable to complete a task in QMC , failing repeatedly after executing at 2 or 3 hrs time , but actually it takes 3 mins only
We are getting the error as below in the log details. Please help us asap , it effecting business.
NoConcatenate
20231219T114729.396+0000 0455 WT_VERSION_2:
20231219T114729.396+0000 0456 LOAD *
20231219T114729.396+0000 0457
20231219T114729.396+0000 0458 ,COD_POWERTRAIN_TYPE as COD_POWERTRAIN_TYPE_ORIGINAL
20231219T114729.396+0000 0459 ,DES_MAKE as DES_MAKE_PWT_TYPE
20231219T114729.396+0000 0460 ,subfield(DES_MODEL, ':',2) as DES_MODEL_PWT_TYPE
20231219T114729.396+0000 0461 ,DES_VERSION as DES_VERSION_PWT_TYPE
20231219T114729.396+0000 0462 ,COD_FUEL as COD_FUEL_PWT_TYPE
20231219T114729.396+0000 0463 ,COD_HYBRID as COD_HYBRID_PWT_TYPE
20231219T114729.396+0000 0464 ,COD_OTHER_FUEL as COD_OTHER_FUEL_PWT_TYPE
20231219T114729.396+0000 0465 ,COD_POWERTRAIN_TYPE as COD_POWERTRAIN_TYPE_ORIGINAL_PWT_TYPE
20231219T114729.396+0000 0466 Resident WT_VERSION
20231219T114729.397+0000 90 fields found: COD_DATASET, COD_VERSION, COD_VERSION_UNIQ, COD_JATO_GLOBAL_SEGMENT, COD_JATO_REGIONAL_SEGMENT, COD_DATA_SOURCE, COD_VOLUME_TYPE, COD_VERSION_TYPE, COD_PRECALC, COD_COUNTRY, COD_MANUF_COUNTRY, COD_MANUF_GROUP, DES_MAKE_GROUP, DES_MODEL_GROUP, COD_MANFTR, DES_MAKE, DES_MODEL, DES_TRIM_LEVEL, DES_LL_MANFTR_GROUP, DES_LL_MANFTR, DES_LL_MAKE_GLBAL, DES_LL_MAKE, DES_LL_MODEL, DES_LL_TRIM, DES_LL_VERSION, DES_LIMITED_VERSION, COD_BODY_STYLE, NUM_DOORS, COD_CAB_STYLE, COD_ROOF_TYPE, NUM_SEATS, COD_FAC_BODY_STYLE, NUM_LITERS, NUM_CC, NUM_CYLINDERS, COD_ENGINE_CONFIG, NUM_KW, NUM_HP, NUM_FISCAL_HP, COD_FUEL, COD_OTHER_FUEL, COD_HYBRID, COD_TURBO, COD_TRANSMISSION_TYPE, COD_TRANSMISSION, NUM_GEAR, COD_DRIVEN_WHEELS, COD_WHEELBASE_TYPE, COD_REAR_WHEELS, NUM_AXELS, NUM_MODEL_YEAR, NUM_MODEL_GENERATION, COD_PLATE, COD_CONV_ROOF, COD_DRIVER_LOC, COD_DATA_SET, FLG_MODEL_MIXED, NUM_LENGHT_MM, NUM_WIDTH_MM, NUM_HEIGHT_MM, NUM_WHEEL_BASE_MM, NUM_LENGHT_IN, NUM_WIDTH_IN, NUM_HEIGHT_IN, NUM_WHEEL_BASE_IN, NUM_LOAD_SPACE_VOLUME, NUM_LOAD_SPACE_CU, COD_DATA_SOURCE_2, COD_COUTRTY_ASSEMBLY, NUM_GVW, NUM_KERB_WEIGHT, NUM_PAYLOAD, NUM_GVW_LBS, NUM_KERB_WEIGHT_LBS, NUM_PAYLOAD_LBS, JOB_ID, COD_JATO_COUNTRY_VERS, COD_SOURCE_DB_VER, FLG_INTERFACE, COD_POWERTRAIN_TYPE, DES_VERSION, KEY_VERSION, COD_POWERTRAIN_TYPE_ORIGINAL, DES_MAKE_PWT_TYPE, DES_MODEL_PWT_TYPE, DES_VERSION_PWT_TYPE, COD_FUEL_PWT_TYPE, COD_HYBRID_PWT_TYPE, COD_OTHER_FUEL_PWT_TYPE, COD_POWERTRAIN_TYPE_ORIGINAL_PWT_TYPE,
20231219T123343.663+0000 1,556,692,963 lines fetched
20231219T123343.677+0000 0467 LEFT JOIN
20231219T123343.677+0000 0468 LOAD DES_MAKE as DES_MAKE_PWT_TYPE
20231219T123343.677+0000 0469 ,replace(DES_MODEL, DES_MAKE&' ', '') as DES_MODEL_PWT_TYPE
20231219T123343.677+0000 0470 ,text(DES_VERSION) as DES_VERSION_PWT_TYPE
20231219T123343.677+0000 0471 ,COD_FUEL as COD_FUEL_PWT_TYPE
20231219T123343.677+0000 0472 ,COD_HYBRID as COD_HYBRID_PWT_TYPE
20231219T123343.677+0000 0473 ,COD_OTHER_FUEL as COD_OTHER_FUEL_PWT_TYPE
20231219T123343.677+0000 0474 ,COD_POWERTRAIN_TYPE_ORIGINAL as COD_POWERTRAIN_TYPE_ORIGINAL_PWT_TYPE
20231219T123343.677+0000 0475 ,if(COD_POWERTRAIN_TYPE_FIXED='', null(),COD_POWERTRAIN_TYPE_FIXED) as COD_POWERTRAIN_TYPE_FIXED
20231219T123343.677+0000 0476 FROM
20231219T123343.677+0000 0477 [..\..\..\..\..\..\..\\QvSourceDocs\ROOT\Data\Received\DataOverRide\PowertrainTypeOverRide.csv]
20231219T123343.677+0000 0478 (txt, utf8, embedded labels, delimiter is ',', msq)
20231219T123343.682+0000 8 fields found: DES_MAKE_PWT_TYPE, DES_MODEL_PWT_TYPE, DES_VERSION_PWT_TYPE, COD_FUEL_PWT_TYPE, COD_HYBRID_PWT_TYPE, COD_OTHER_FUEL_PWT_TYPE, COD_POWERTRAIN_TYPE_ORIGINAL_PWT_TYPE, COD_POWERTRAIN_TYPE_FIXED,
20231219T123345.835+0000 1,819,260 lines fetched
20231219T123345.835+0000 Joining/Keeping
20231219T125610.062+0000 -129 =
20231219T125727.851+0000 Execution Failed
20231219T125727.946+0000 Execution finished.
-129 is typically an out-of-memory error.
Hi @KishoreSuriF,
I agree with @Or's post. In addition to the article he provided, this recent forum post dealt with the same error: Solved: QMC Task is not loaded successfully - Qlik Community - 2143298.
Best Regards
Your are loading 1.5 billions records from a resident table with 90 fields and are joining then millions of records against it. It's not surprising that you are going out of RAM.
The simplest approach might be to store the first table as qvd and drop it and then loading the join against the qvd-data. It should reduce the RAM consumption significantly. But in a longer term you should consider to implement any incremental logic.
After following multiple approaches, we didn’t succeed.
so, we restored the back dated files and QVDs and reloaded successfully.Now everything is going good. If anything required we will get back to you
-129 is typically an out-of-memory error.
Hi @KishoreSuriF,
I agree with @Or's post. In addition to the article he provided, this recent forum post dealt with the same error: Solved: QMC Task is not loaded successfully - Qlik Community - 2143298.
Best Regards
Your are loading 1.5 billions records from a resident table with 90 fields and are joining then millions of records against it. It's not surprising that you are going out of RAM.
The simplest approach might be to store the first table as qvd and drop it and then loading the join against the qvd-data. It should reduce the RAM consumption significantly. But in a longer term you should consider to implement any incremental logic.
Thanks for your responses.
We reached our team to increase resource size.
Sorry, we still facing the issue even after RAM increase.
with suggestions from our Qlik consultant, we made some changes in qlik script, due to Christmas holidays, we are unable to confirm now
will check and update on tomorrow
Hi @KishoreSuriF,
Just increasing the RAM may not be enough. As @marcus_sommer pointed out, you are loading 1.5 billions records from a resident table with 90 fields and are joining then millions of records against it.
You likely need to revisit the load script and implement the suggestion he made previously.
Remember that memory is finite.
Best Regards
Thanks for your reply's.
We have followed the instructions as mentioned above like creating QVDs and then use that QVDs . But still same issue . It occupies complete 100% memory and automatically shuts down the server system now.
You are loading too much data into the app and then joining millions of rows against it. Suggest you contact your account manager who can assist you in scheduling an engagement with Qlik Consulting to assist in developing your application.
Best Regards
Sure, thanks for your suggestion.
We already reached them and they also suggested the same to create QVDs and check the issue.