Incremental load implement

[sanjeeva_279]

Hi I am trying to implement incremental load for one of my dashbaord

issues : DATE' is not a recognized built-in function name/date is not comparable with INT 

i have loaded tables data in qvd and identified date field ,date field i am using it as variable max(datefield) for max date data

when i am trying to concatenate data i am having issues with date and int issues

QVD: name,date,salary     ---date as field for date  max(date) as maxdate

Incremental : 

select name ,date,salary from table employee  where date > maxdate;

concatenate

load  name,date,salary from qvd;

in this scenario i am getting date issues as maxdate and date field is not compatable  etc...

[jyothish8807]

try this:

let maxdate= Date(max(Date));  // Make sure both Date field and output of variable "maxdate" is in same format

Incremental : 

select name ,date,salary from table employee  where date > $(maxdate);

concatenate

load  name,date,salary from qvd;

[sanjeeva_279]

THANKS FOR THE MAIL,STILL I SEE ERROR  AS 

":'Date' is not a recognized built-in function name."

even i kept both the formats same i see the issues of  date function or incompatability

Incremental : 

select name ,date,salary from table employee  where date > $(maxdate);

concatenate

load  name,date,salary from qvd;

seems selecting fields from table and loading fileds from tables for incremental use has issues

[jyothish8807]

Hi Sanjeeva,

Can you try this, i guess the issue is because the the first table is coming directly from DB.

Incremental : 

Load *

where date > $(maxdate);

select name ,date,salary from table employee ;

concatenate

load  name,date,salary from qvd;

[PrashantSangle]

can you post your script which you are trying? It will help us to provide more accurate solution.

[jonathandienst · Accepted solution]

maxdate will not be evaluated so you are trying to inject the text of the expression which does not work. You need to calculate the max in a load and then peek() the value to get the max date into the variable - something like this:

MaxDate:
LOAD Max(date) as MaxDate
Resident ....;

Let vMaxDate = Date(Peek('MaxDate));
Drop table MaxDate;

Incremental : 
select name, date, salary 
from employee 
where date > '$(vMaxDate)'; 

concatenate(Incremental)
load name,date,salary 
from file.qvd (qvd);

[jyothish8807]

Try this :

A:

Load

Date(Max(Date) ) as MaxDate

from employee;

Let  maxdate= peek('MaxDate',0,'A');

Drop table A;

Incremental : 

Load *

where date > $(maxdate);

select name ,date,salary from table employee ;

concatenate

load  name,date,salary from qvd;

[sanjeeva_279]

Thanks all for the help,it worked

i am missing  

where date > '$(vMaxDate)';

i am missing '' /text for the $(vmaxdate)