Skip to main content
Announcements
SYSTEM MAINTENANCE: Thurs., Sept. 19, 1 AM ET, Platform will be unavailable for approx. 60 minutes.
cancel
Showing results for 
Search instead for 
Did you mean: 
Tayaba
Contributor
Contributor

Incremental Load using if and else statements

I am trying to develop an incremental load script using if and else statements. First, i am check to see if the qvd file exist and if the qvd file exist then i am getting the max_id from that qvd file. I am creating a variable called MAXID so i can use it in my original data source. Then i am connecting to the original data source and pulling all the records that are greater than the MAXID form the qvd file. Lastly, i concatenate the qvd file with the new record and store in the file. The problem is that i am getting error when i try to run my script. Can someone please help me on what i am doing wrong here. I am getting an error saying that it doesn't recognize the 'then' statements. 

if NOT IsNull(FileSize('lib://Reports/Data_file.qvd')>0)

then

NoConcatenate;

MaxTable:

Load

max("WorkflowID") as MaxID

FROM [lib://Reports/Data_file.qvd]

(qvd);

Let MAXID = num(peek('MaxID',0,MaxTable));

drop table MaxTable;

LIB CONNECT TO 'Original_database';

NewRecordData:

SQL SELECT WorkflowID,

    ActiveID,

    Activitynumber

FROM Knowdatabase..workflowtable where

workflowid > $(MAXID);

Concatenate

Load * from 'lib://Reports/Data_file.qvd' (qvd);

STORE NewRecordData into 'lib://FC_KYC_Reports/PBAData.qvd' (qvd);

exit Script;

Else

NoConcatenate;

LIB CONNECT TO 'Original_database';

OriginalData:

SQL SELECT WorkflowID,

    ActiveID,

    Activitynumber

FROM FROM Knowdatabase..workflowtable;

Store PBAData into 'lib://Reports/Data_file.qvd' (qvd);

Exit Script;

 

 

Labels (4)
1 Reply
vinieme12
Champion III
Champion III

As below

// (filesize > 0 ) this will either be true or false; So checking Isnull(true/false) isn't doing anything

if FileSize('lib://Reports/PBAData.qvd')>0 THEN 

//Load existing data first and then load max date as this would be faster since data is already in memory

PBAData:
Load
*
FROM [lib://Reports/PBAData.qvd] (qvd);

MaxTable:
Load max("WorkflowID") as MaxID
Resident PBAData;

Let MAXID = num(peek('MaxID',0,MaxTable));
drop table MaxTable;

LIB CONNECT TO 'Original_database';


Concatenate(PBAData)
//NewRecordData

SQL SELECT WorkflowID,

ActiveID,

Activitynumber

FROM Knowdatabase.workflowtable where

workflowid > $(MAXID);

STORE PBAData into 'lib://FC_KYC_Reports/PBAData.qvd' (qvd);

 

Else

 

LIB CONNECT TO 'Original_database';

PBAData:

SQL SELECT WorkflowID,

ActiveID,

Activitynumber

FROM Knowdatabase.workflowtable;

Store PBAData into 'lib://FC_KYC_Reports/PBAData.qvd' (qvd);

End IF

 

Also don't think you need to maintain a second Qvd in this case ie PBAData.qvd and Date_file.qvd ; 

you can just keep concatenating to PBAData.qvd or create PBAData.qvd if it doesn't exists

Vineeth Pujari
If a post helps to resolve your issue, please accept it as a Solution.