Help with first go at incremental loads and updates
Ok, I am working on improving our dataset from the last 3 full months to last 12 full months of incidents and decided to go the incremental load and update route. I am struggling with a few things though.
After the initial load I added the below code to my WHERE statement for Inc_Table query.
and incident.data_updated >= '$(LastExecTime)'
Then I added the below concatenate between Inc_Table populate and STORE (before store in order to concat old records where incident_number doesn't exist in the new Inc_Table, correct?)
So, where do I need help:
The Inc_Table query is fetching a number of rows that is no where near what it should. I've tested the query with a manually typed LastExecTime using DBArtisan and I get ~40 rows which is accurate but Qlikview always returns ~1,300 lines fetched. No idea what is going on with this. Assuming something with LastExecTime.
How do I change my data_updated to query max data_updated from QVD and is that a better way to do it?
Also, is the method I am using correct to query newly updated records and merge with QVD records?