Skip to main content
Announcements
Introducing Qlik Answers: A plug-and-play, Generative AI powered RAG solution. READ ALL ABOUT IT!
cancel
Showing results for 
Search instead for 
Did you mean: 
SaurabhFirke
Partner - Contributor
Partner - Contributor

Why is the total number of entries the same after performing an incremental load with new data?

When applying a load, it seems that the system is fetching the wrong number of data. Upon fetching new data today, it was discovered that there were 312 new entries specifically for today's date. The total number of entries in the dataset before the load was 636,058. However, upon running the incremental load again, it fetched only 315 new records for today's date, but the total number of entries remained the same at 636,058. This raises the question of why the total number of entries did not increase despite fetching additional records for today

 

Case 1Case 1

 

 

Case 2Case 2

 

 

Labels (4)
1 Reply
SaurabhFirke
Partner - Contributor
Partner - Contributor
Author

This incremental load script I am using.

 

 

 

LIB CONNECT TO 'SapSql_Prod';

EBAN_Max_Date:
Load max(ERDAT) as Max_Date
from [$(vQVDPath)EBAN.qvd](qvd);
STORE * FROM [EBAN_Max_Date] INTO [$(vQVDPath)EBAN_Max_Date.qvd];


Let Last_Updated_date = date(Peek('MaxDate', 0, 'EBAN_Max_Date'),'YYYYMMDD');

trace $(Last_Updated_date);

Incremental:
load*, BANFN&'-'&BNFPO as KEY;
Select
BANFN,
ERNAM,
ERDAT,
BNFPO,
FROM EBAN where ERDAT >= '$(Last_Updated_date)';


Concatenate
LOAD
KEY,
BANFN,
ERNAM,
ERDAT,
BNFPO
FROM [$(vQVDPath)EBAN.qvd](qvd)
Where not Exists(KEY);


store Incremental into [$(vQVDPath)EBAN.qvd](qvd);

drop table Incremental;
drop table EBAN_Max_Date;


Exit Script;