Qlikview task fails in QMC

[KishoreSuriF]

Hi All,

We are unable to complete a task in QMC , failing repeatedly after executing at 2 or 3 hrs time , but actually it takes 3 mins only

We are getting the error as below in the log details. Please help us asap , it effecting business.

NoConcatenate
  20231219T114729.396+0000 0455     WT_VERSION_2:
  20231219T114729.396+0000 0456     LOAD *
  20231219T114729.396+0000 0457               
  20231219T114729.396+0000 0458          ,COD_POWERTRAIN_TYPE                         as COD_POWERTRAIN_TYPE_ORIGINAL 
  20231219T114729.396+0000 0459          ,DES_MAKE                               as DES_MAKE_PWT_TYPE
  20231219T114729.396+0000 0460          ,subfield(DES_MODEL, ':',2)                      as DES_MODEL_PWT_TYPE
  20231219T114729.396+0000 0461          ,DES_VERSION                             as DES_VERSION_PWT_TYPE
  20231219T114729.396+0000 0462          ,COD_FUEL                               as COD_FUEL_PWT_TYPE
  20231219T114729.396+0000 0463          ,COD_HYBRID                             as COD_HYBRID_PWT_TYPE
  20231219T114729.396+0000 0464          ,COD_OTHER_FUEL                           as COD_OTHER_FUEL_PWT_TYPE
  20231219T114729.396+0000 0465          ,COD_POWERTRAIN_TYPE                         as COD_POWERTRAIN_TYPE_ORIGINAL_PWT_TYPE
  20231219T114729.396+0000 0466     Resident WT_VERSION
  20231219T114729.397+0000        90 fields found: COD_DATASET, COD_VERSION, COD_VERSION_UNIQ, COD_JATO_GLOBAL_SEGMENT, COD_JATO_REGIONAL_SEGMENT, COD_DATA_SOURCE, COD_VOLUME_TYPE, COD_VERSION_TYPE, COD_PRECALC, COD_COUNTRY, COD_MANUF_COUNTRY, COD_MANUF_GROUP, DES_MAKE_GROUP, DES_MODEL_GROUP, COD_MANFTR, DES_MAKE, DES_MODEL, DES_TRIM_LEVEL, DES_LL_MANFTR_GROUP, DES_LL_MANFTR, DES_LL_MAKE_GLBAL, DES_LL_MAKE, DES_LL_MODEL, DES_LL_TRIM, DES_LL_VERSION, DES_LIMITED_VERSION, COD_BODY_STYLE, NUM_DOORS, COD_CAB_STYLE, COD_ROOF_TYPE, NUM_SEATS, COD_FAC_BODY_STYLE, NUM_LITERS, NUM_CC, NUM_CYLINDERS, COD_ENGINE_CONFIG, NUM_KW, NUM_HP, NUM_FISCAL_HP, COD_FUEL, COD_OTHER_FUEL, COD_HYBRID, COD_TURBO, COD_TRANSMISSION_TYPE, COD_TRANSMISSION, NUM_GEAR, COD_DRIVEN_WHEELS, COD_WHEELBASE_TYPE, COD_REAR_WHEELS, NUM_AXELS, NUM_MODEL_YEAR, NUM_MODEL_GENERATION, COD_PLATE, COD_CONV_ROOF, COD_DRIVER_LOC, COD_DATA_SET, FLG_MODEL_MIXED, NUM_LENGHT_MM, NUM_WIDTH_MM, NUM_HEIGHT_MM, NUM_WHEEL_BASE_MM, NUM_LENGHT_IN, NUM_WIDTH_IN, NUM_HEIGHT_IN, NUM_WHEEL_BASE_IN, NUM_LOAD_SPACE_VOLUME, NUM_LOAD_SPACE_CU, COD_DATA_SOURCE_2, COD_COUTRTY_ASSEMBLY, NUM_GVW, NUM_KERB_WEIGHT, NUM_PAYLOAD, NUM_GVW_LBS, NUM_KERB_WEIGHT_LBS, NUM_PAYLOAD_LBS, JOB_ID, COD_JATO_COUNTRY_VERS, COD_SOURCE_DB_VER, FLG_INTERFACE, COD_POWERTRAIN_TYPE, DES_VERSION, KEY_VERSION, COD_POWERTRAIN_TYPE_ORIGINAL, DES_MAKE_PWT_TYPE, DES_MODEL_PWT_TYPE, DES_VERSION_PWT_TYPE, COD_FUEL_PWT_TYPE, COD_HYBRID_PWT_TYPE, COD_OTHER_FUEL_PWT_TYPE, COD_POWERTRAIN_TYPE_ORIGINAL_PWT_TYPE, 
  20231219T123343.663+0000      1,556,692,963 lines fetched
  20231219T123343.677+0000 0467 LEFT JOIN
  20231219T123343.677+0000 0468     LOAD DES_MAKE                               as DES_MAKE_PWT_TYPE
  20231219T123343.677+0000 0469          ,replace(DES_MODEL, DES_MAKE&' ', '')                 as DES_MODEL_PWT_TYPE
  20231219T123343.677+0000 0470          ,text(DES_VERSION)                           as DES_VERSION_PWT_TYPE
  20231219T123343.677+0000 0471          ,COD_FUEL                               as COD_FUEL_PWT_TYPE
  20231219T123343.677+0000 0472          ,COD_HYBRID                             as COD_HYBRID_PWT_TYPE
  20231219T123343.677+0000 0473          ,COD_OTHER_FUEL                           as COD_OTHER_FUEL_PWT_TYPE
  20231219T123343.677+0000 0474          ,COD_POWERTRAIN_TYPE_ORIGINAL                     as COD_POWERTRAIN_TYPE_ORIGINAL_PWT_TYPE
  20231219T123343.677+0000 0475         ,if(COD_POWERTRAIN_TYPE_FIXED='', null(),COD_POWERTRAIN_TYPE_FIXED) as COD_POWERTRAIN_TYPE_FIXED
  20231219T123343.677+0000 0476     FROM
  20231219T123343.677+0000 0477     [..\..\..\..\..\..\..\\QvSourceDocs\ROOT\Data\Received\DataOverRide\PowertrainTypeOverRide.csv]
  20231219T123343.677+0000 0478     (txt, utf8, embedded labels, delimiter is ',', msq)
  20231219T123343.682+0000        8 fields found: DES_MAKE_PWT_TYPE, DES_MODEL_PWT_TYPE, DES_VERSION_PWT_TYPE, COD_FUEL_PWT_TYPE, COD_HYBRID_PWT_TYPE, COD_OTHER_FUEL_PWT_TYPE, COD_POWERTRAIN_TYPE_ORIGINAL_PWT_TYPE, COD_POWERTRAIN_TYPE_FIXED, 
  20231219T123345.835+0000      1,819,260 lines fetched
  20231219T123345.835+0000      Joining/Keeping
  20231219T125610.062+0000      -129 = 
  20231219T125727.851+0000      Execution Failed
  20231219T125727.946+0000      Execution finished.

[Or · Accepted solution]

-129 is typically an out-of-memory error.

https://community.qlik.com/t5/Official-Support-Articles/Error-129-or-128-when-reloading-app/ta-p/171...

[Chip_Matejowsky · Accepted solution]

Hi @KishoreSuriF,

I agree with @Or's post. In addition to the article he provided, this recent forum post dealt with the same error:  Solved: QMC Task is not loaded successfully - Qlik Community - 2143298.

Best Regards

[marcus_sommer · Accepted solution]

Your are loading 1.5 billions records from a resident table with 90 fields and are joining then millions of records against it. It's not surprising that you are going out of RAM.

The simplest approach might be to store the first table as qvd and drop it and then loading the join against the qvd-data. It should reduce the RAM consumption significantly. But in a longer term you should consider to implement any incremental logic.

[KishoreSuriF]

Thanks for your responses.

We reached our team to increase resource size.

[KishoreSuriF]

Sorry, we still facing the issue even after RAM increase.

with suggestions from our Qlik consultant, we made some changes in qlik script, due to Christmas holidays, we are unable to confirm now

will check and update on tomorrow 

[Chip_Matejowsky]

Hi @KishoreSuriF,

Just increasing the RAM may not be enough. As @marcus_sommer  pointed out, you are loading 1.5 billions records from a resident table with 90 fields and are joining then millions of records against it. 

You likely need to revisit the load script and implement the suggestion he made previously.

Remember that memory is finite.

Best Regards

[KishoreSuriF]

Thanks for your reply's.

We have followed the instructions as mentioned above like creating QVDs and then use that QVDs . But still same issue . It occupies complete 100% memory and automatically shuts down the server system now.

[Chip_Matejowsky]

You are loading too much data into the app and then joining millions of rows against it. Suggest you contact your account manager who can assist you in scheduling an engagement with Qlik Consulting to assist in developing your application.

Best Regards 

[KishoreSuriF]

Sure, thanks for your suggestion. 

We already reached them and they also suggested the same to create QVDs and check the issue.