incremental load implementation

[sakshikaul]

Hi 

I want to implement increment load. current scenario is as follows :-

I have full load file and source is FTP, and All incremental Files are also created on daily basis in same folder where full load is kept(source  ftp). All incremental file only contains the changed record not the full record. earlier I handed the same by concatenating all the files like below:-

Following are the folders on FTP:-

Material_Master_full:
LOAD distinct MATERIAL,
COST,
MRP,
ChangeDt
FROM
[ftp://userid:userpass@10.47.77.207/IFR/IAP/QLK/MAT_MAS/MAT_MAS_full_000009.txt]
(txt, codepage is 1252, embedded labels, delimiter is '|', msq);

concatenate(Material_Master_full)

Material_Master_full:
LOAD distinct MATERIAL,

COST,
MRP,
ChangeDt
FROM
[ftp://userID:userPass@10.47.77.207/IFR/IAP/QLK/MAT_MAS/MAT_MAS_20190108_000012.txt]
(txt, codepage is 1252, embedded labels, delimiter is '|', msq);

but since there are files which are created on daily basis so its not feasible to implement the same like this.

so what's the other way out to implement the same (Incremental load ). Please Help?

[sakshikaul]

please help  Marcus_sommer 

[felipedl]

You could use @rwunderlich DoDir.qvs script (available on git https://github.com/QlikDeploymentFramework/Qlik-Deployment-Framework/releases/tag/v.1.7.2), to get all files on your ftp and loop around it, since you don't know specifically what name your file could take.

Next on, you'd have to check which files you'd already read (could be a simple qvd saying which file you read with a flag for example) and do whatever requirement you need next.

[sakshikaul]

Hi 

File name (nominclature ) is same for the files which I will get as incremental load