Incremental Load

[ioannagr]

Hello.

So this is the first time I'm trying to use  Incremental Data Load, based on a date field.

This is the code I've used:

1.Initial Load of the data of the table i want to use  from the database . In this table I have the date field "updated"

Questionnaire:

LOAD * from

SQL SELECT *
FROM "public"."questionnaire";

//Store in a QVD
STORE Questionnaire INTO '$(vPathQVD)/Questionnaire.qvd';

drop table Questionnaire;

//3.MaxUpdateDate

MaxQuestionnaireUpdateLoad:
Load Max(updated) as MaxUpdate
from [$(vPathQVD)Questionnaire.qvd] (qvd);

//4.Store in a variable

Let MaxUpdate = Peek('MaxUpdate', 0 ,MaxErwtimatologioUpdateLoad);

//5.Get the new dates from "Questionnaire" : ---> Here, which might be my mistake, i'm using the exact same path and table as above, with a sole difference at the where statement at the bottom. 

QuestionnaireNEW:
NoConcatenate

LOAD *

SQL SELECT *
FROM "public"."questionnaire"
where updated>$(MaxUpdate) ;

//6. Concatenate old with new

 Concatenate (QuestionnaireNEW)
Load * from [$(vPathQVD)Questionnaire.qvd] (qvd);

//7. store concatenation in qvd
Store QuestionnaireNEW into [$(vPathQVD)Questionnaire.qvd] (qvd);

I'm getting the error, cant find  "id2" in QuestionnaireNEW, while Questionnaire runs perfectly fine and is the same as QuestionnaireNEW.

I need your ideas.

Help, i've been trying for hours 😕

[shivanisapkale]

Hello,

Is id2 present in this script

Questionnaire:

LOAD * from

SQL SELECT *
FROM "public"."questionnaire";

as well as in this table QuestionnaireNEW:

Regards,

Shivani Sapkale

[ioannagr]

Hello, 

yes they are in fact the same table , only for QuestionnaireNew I have a where clause to get the new entries.

Could it be that SQL doesn't understand the $() variable format that Qlik has?

[thi_pham]

Please try to put the variable in a single quote. I made this mistake many times. 

QuestionnaireNEW:
NoConcatenate

LOAD *

SQL SELECT *
FROM "public"."questionnaire"
where updated>'$(MaxUpdate) ';

[shivanisapkale]

Hi,

Check what value is coming in your variable.Is it null or any value is being passed.

Check the date format of variable and in your database.

Regards,

Shivani Sapkale