topic Re: incremental load implementation in QlikView

incremental load implementation

sakshikaul — Wed, 25 Nov 2020 16:16:04 GMT

I want to implement increment load. current scenario is as follows :-

I have full load file and source is FTP, and All incremental Files are also created on daily basis in same folder where full load is kept(source ftp). All incremental file only contains the changed record not the full record. earlier I handed the same by concatenating all the files like below:-

Following are the folders on FTP:-

Material_Master_full:
LOAD distinct MATERIAL,
COST,
MRP,
ChangeDt
FROM
[ftp://userid:userpass@10.47.77.207/IFR/IAP/QLK/MAT_MAS/MAT_MAS_full_000009.txt]
(txt, codepage is 1252, embedded labels, delimiter is '|', msq);

concatenate(Material_Master_full)

Material_Master_full:
LOAD distinct MATERIAL,

COST,
MRP,
ChangeDt
FROM
[ftp://userID:userPass@10.47.77.207/IFR/IAP/QLK/MAT_MAS/MAT_MAS_20190108_000012.txt]
(txt, codepage is 1252, embedded labels, delimiter is '|', msq);

but since there are files which are created on daily basis so its not feasible to implement the same like this.

so what's the other way out to implement the same (Incremental load ). Please Help?

Re: incremental load implementation

sakshikaul — Tue, 08 Jan 2019 07:14:29 GMT

please help Marcus_sommer

Re: incremental load implementation

felipedl — Tue, 08 Jan 2019 12:46:33 GMT

You could use @rwunderlich DoDir.qvs script (available on git https://github.com/QlikDeploymentFramework/Qlik-Deployment-Framework/releases/tag/v.1.7.2), to get all files on your ftp and loop around it, since you don't know specifically what name your file could take.

Next on, you'd have to check which files you'd already read (could be a simple qvd saying which file you read with a flag for example) and do whatever requirement you need next.

Re: incremental load implementation

sakshikaul — Wed, 09 Jan 2019 05:18:54 GMT

File name (nominclature ) is same for the files which I will get as incremental load