Incremental Load Question

[Hareeshkommineni]

While working on incremental load, how can we retrieve the last record 'data' of qvd to insert new data?

[groveliam]

This link should help you: Re: Incremental Load

[groveliam]

Just change max date to whatever you are trying to retrieve.

[MarioCenteno]

try

LET vQVDexists = NOT ISNULL(QVDCreateTime('lib://blahblah/v3\ExpediteAuditExtract.qvd'));    
LET vNow = Now();  
IF $(vQVDexists) THEN;  
TRACE('QVD Exists, so we are going to append');  
    LatestUpdate:    
    LOAD  Date(Max(updated_dt)) as MaxDate  
    FROM [lib://blahblah/v3\ExpediteAuditExtract.qvd]  
    (qvd)  
    ;  
Trace('Got to the LatestUpdate');  
      
    LET vMaxDate = Peek('MaxDate',0,'LatestUpdate');  
    TRACE('Max update' & '$(vMaxDate)');  
      
    NewData:  
    LOAD *  
    ;  
    SQL SELECT stuff  
    FROM "TableOfInterest"  
    WHERE updated_dt >  '$(vMaxDate)'  
     ;  
  
  
Trace('Concatenating the new data with the old data');  
    Concatenate(NewData)  
    LOAD * FROM [lib://blahblah/v3\ExpediteAuditExtract.qvd]  
    (qvd)  
    ;  
    STORE NewData into [lib://blahblah/v3\ExpediteAuditExtract.qvd] (qvd);  
    DROP Tables NewData, LatestUpdate  
    ;  
  ELSE  
  Trace('QVD does not exist, so we are doing a full load');  
    DATA:  
    LOAD everything...  
;  
SQL SELECT everything FROM...  
;  
  
  
    STORE DATA into [lib://blahblah/v3\ExpediteAuditExtract.qvd] (qvd);  
    DROP Table DATA  
    ;  
END IF  
;