topic Incremental Load Script in App Development

Incremental Load Script

danaleota1 — Thu, 18 Nov 2021 14:34:06 GMT

I am trying load a qvd incrementally and get an error when loading the view at the where clause. I was trying to load only dates that come after the last date timestamp from the qvd. Any suggestions on why its not working would be great thanks!

[MaxKeyLoad]:
Load max(INTRACTN_TS) as vMAXKey
from [lib://PROD_QlikData/abc.qvd](qvd);

let vMAXKey = peek('INTRACTN_TS',0, MaxKeyLoad);

LIB CONNECT TO 'D L_PROD';

[ARC_NEXIDIA_INTRACTN]:
trim(INTRACTN_ID) as InteractionId,
INTRACTN_TS,
CRSSTLK_SGMNT_CNT;

[ARC_NEXIDIA_INTRACTN]:
SELECT
"INTRACTN_ID",
"INTRACTN_TS",
"CRSSTLK_SGMNT_CNT"
FROM "PROD_S_D"."ARC_NEXIDIA_INTRACTN"
where "INTRACTN_TS" > '$(vMAXKey)'
;

Re: Incremental Load Script

mdmukramali — Thu, 18 Nov 2021 15:56:30 GMT

Dear,

The error is in the field name.

check below script.

[MaxKeyLoad]:
// Load max(INTRACTN_TS) as vMAXKey

Load max(INTRACTN_TS) as Max_INTRACTN_TS
from [lib://PROD_QlikData/abc.qvd](qvd);

let vMAXKey = peek('Max_INTRACTN_TS',0, MaxKeyLoad);

LIB CONNECT TO 'D L_PROD';

[ARC_NEXIDIA_INTRACTN]:
trim(INTRACTN_ID) as InteractionId,
INTRACTN_TS,
CRSSTLK_SGMNT_CNT;

[ARC_NEXIDIA_INTRACTN]:
SELECT
"INTRACTN_ID",
"INTRACTN_TS",
"CRSSTLK_SGMNT_CNT"
FROM "PROD_S_D"."ARC_NEXIDIA_INTRACTN"
where "INTRACTN_TS" > '$(vMAXKey)'
;

Re: Incremental Load Script

danaleota1 — Thu, 18 Nov 2021 16:45:45 GMT

Hi yes d'oh thank you for pointing that out! That worked in addition to adjusting my where clause to:

where to_number(to_char("INTRACTN_TS",'YYYY-MM-DD HH:MM:SS')) > '$(vMAXKey)'
;