This content has been marked as final. Show 1 reply
Ok, I am working on improving our dataset from the last 3 full months to last 12 full months of incidents and decided to go the incremental load and update route. I am struggling with a few things though.
First, I did the initial load:
Inc_Table: SELECT incident.number as incident_number, incident.data_updated as data_updated FROM tbl_incident incident WHERE incident.opened_at >= convert(varchar(10),dateadd(mm,-12,dateadd(dd,-(day(getdate())-1),getdate())),101); STORE Inc_Table INTO incidents.QVD; LET LastExecTime = now();
After the initial load I added the below code to my WHERE statement for Inc_Table query.
and incident.data_updated >= '$(LastExecTime)'
Then I added the below concatenate between Inc_Table populate and STORE (before store in order to concat old records where incident_number doesn't exist in the new Inc_Table, correct?)
Concatenate LOAD incident_number, data_updated FROM [incidents.QVD] (qvd) WHERE NOT EXISTS(incident_number);
So, where do I need help: