I created an incremental load script however it is not adding the new (incremental) data to my QVD file. I have first connected to DB and copy the full data to QVD file. Subsequently I load from the QVD file and only select the new data from DB to be added to the QVD file.
// Loading data from QVD SRO: LOAD [Form Ref No], [Form Name], CompletionDate FROM [lib://GS_Folder/SRO.qvd] (qvd);
// Find last modified date Last_Updated_Date: load max (CompletionDate) as Maxdate resident SRO;
// Store last modified date to a variable Let Last_Updated_Date=peek('Maxdate',0,'Last_Updated_Date');
// Delete Table SRO Drop table SRO;
// Load new and incremental data from DB LOAD [Form Ref No], [Form Name], CompletionDate;
[Incremental]: SELECT "Form Ref No", "Form Name", CompletionDate FROM DEA.COC.vwSROHeader where CompletionDate > $(Last_Updated_Date);
// Concatenate with QVD Concatenate LOAD [Form Ref No], [Form Name], CompletionDate FROM [lib://GS_Folder/SRO.qvd] (qvd);
// Replace old QVD file Store Incremental into [lib://GS_Folder/SRO.qvd] (qvd);
I think the error is due to CompletionDate which is in the format of 21/11/2019 5:06:29 PM. Tried adding "LOAD TimeStamp(TimeStamp#(CompletionDate,'DD/MM/YYYY hh:mm:ss')) as CompletionDate," but it gave an error. Any idea?
Hopefully reviewing that may help you find the problem, if you get further info, shout back. My post will also kick things back up in the list, so someone else may see things and be able to add something else too.
To help users find verified answers, please do not forget to use the "Accept as Solution" button on any post(s) that helped you resolve your problem or question. I now work a compressed schedule, Tuesday, Wednesday and Thursday, so those will be the days I will reply to any follow-up posts.