Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
user8
Contributor III
Contributor III

Incremental Load script

Hi,

I created an incremental load script however it is not adding the new (incremental) data to my QVD file. I have first connected to DB and copy the full data to QVD file. Subsequently I load from the QVD file and only select the new data from DB to be added to the QVD file. 

Load Script

// Loading data from QVD
SRO:
LOAD
[Form Ref No],
[Form Name],
CompletionDate
FROM [lib://GS_Folder/SRO.qvd] (qvd);

// Find last modified date
Last_Updated_Date:
load max (CompletionDate) as Maxdate
resident SRO;

// Store last modified date to a variable
Let Last_Updated_Date=peek('Maxdate',0,'Last_Updated_Date');

// Delete Table SRO
Drop table SRO;

// Load new and incremental data from DB
LOAD
[Form Ref No],
[Form Name],
CompletionDate;

[Incremental]:
SELECT
"Form Ref No",
"Form Name",
CompletionDate
FROM DEA.COC.vwSROHeader where CompletionDate > $(Last_Updated_Date);

// Concatenate with QVD
Concatenate
LOAD
[Form Ref No],
[Form Name],
CompletionDate
FROM [lib://GS_Folder/SRO.qvd] (qvd);

// Replace old QVD file
Store Incremental into [lib://GS_Folder/SRO.qvd] (qvd);

// Drop Incremental Table
Drop table Incremental;

Load Status

Started loading data
SRO << SRO
(QVD (row-based) optimized)
Lines fetched: 1,154
Last_Updated_Date << SRO
Lines fetched: 1
Incremental << vwSROHeader
Lines fetched: 0
Incremental << SRO
(QVD (row-based) optimized)
Lines fetched: 1,154
Creating search index
Search index creation completed successfully
App saved
Finished successfully
0 forced error(s)
0 synthetic key(s)

Below are the 2 new records I want to add to QVD from DB

Form Ref NoForm NameCompletionDate
OSFT1911-00017COC - Form21/11/2019 5:06:29 PM
CHQR1911-00033COC - Form221/11/2019 5:17:39 PM

 

Any help is most appreciated. Thank you!

Labels (1)
2 Replies
user8
Contributor III
Contributor III
Author

I think the error is due to CompletionDate which is in the format of 21/11/2019 5:06:29 PM.
Tried adding "LOAD TimeStamp(TimeStamp#(CompletionDate,'DD/MM/YYYY hh:mm:ss')) as CompletionDate," but it gave an error. Any idea?

Brett_Bleess
Former Employee
Former Employee

Best I have is Help link for you, there is mention in there of needing to ensure you have the correct timestamp format per the backend DB, that may be where things are going wrong?

https://help.qlik.com/en-US/qlikview/April2019/Subsystems/Client/Content/QV_QlikView/QVD_Incremental...

Hopefully reviewing that may help you find the problem, if you get further info, shout back.  My post will also kick things back up in the list, so someone else may see things and be able to add something else too.

Regards,
Brett

To help users find verified answers, please do not forget to use the "Accept as Solution" button on any post(s) that helped you resolve your problem or question.
I now work a compressed schedule, Tuesday, Wednesday and Thursday, so those will be the days I will reply to any follow-up posts.