Qlik Community

New to Qlik Sense

Discussion board where members can get started with Qlik Sense.

alexvalqlik
New Contributor II

Script to optimize data load

Good day,

Please show your support, the following script takes an average of 00:04:30 to make the load of 90 new records in a table of SAP, with the index 'VBELN' field need to reduce optimize the same time. Thank you.

SET vQVD_File = 'LIB://Qvd_Ventas/ZVTA_DIMEN_CAB.qvd';

Temp_Tabla:

NoConcatenate

LOAD Distinct VBELN From $(vQVD_File) (qvd);

zvta_dimencab_new:

NoConcatenate

LOAD VBELN, FKART, VSTEL, PSTYV, VBTYP

where not Exists(VBELN) ;      

SELECT VBELN, FKART, VSTEL, PSTYV, VBTYP

  FROM ZVTA_DIMEN_CAB ;

If ScriptErrorCount = 0 then

   LOAD VBELN, FKART, VSTEL, PSTYV, VBTYP

   From $(vQVD_File) (qvd)    ;

   STORE * FROM [zvta_dimencab_new] INTO 'LIB://Qvd_Ventas/ZVTA_DIMEN_CAB.qvd';

    DROP TABLE [zvta_dimencab_new];

End if

;

DROP TABLE [Temp_Tabla]

;

4 Replies
dcj
Valued Contributor II

Re: Script to optimize data load

Hi Alex,

   The correct troubleshooting should be understand where and what moment the problem happens. The tool to help on this situations is debug mode executing your code step by step.

   By the way, there are a lot of environment configurations related.

   My other sugestion is slicing you code, executing once and analyze the prior step.

Regards,

Ricardo Gerhard

Re: Script to optimize data load

You are selecting every row from the ZVTA_DIMEN_CAB table and then filtering with the LOAD statement. If the ZVTA_DIMEN_CAB table is large, this could take some time.

You should modify your SQL SELECT to include a WHERE clause that SELECTs row only greater than the max modification date in the QVD (or a greater sequence number). It's hard to give you specifics without knowing your table columns, but search for "incremental load" for examples.

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

alexvalqlik
New Contributor II

Re: Script to optimize data load

Thank you very much

alexvalqlik
New Contributor II

Re: Script to optimize data load

Thank you very much