Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hello experts,
so basically my question is described in the subject of this message, I am asked to perform hard delete incremental loading script without having a proper lastupdate date field in the source data.
The thing is on the UI application (not qlik sense app), they can delete and/or disable (which to me counts as an update) an id, but the source data don't have such a date field that I can use.
Any ideas on how I should handle this?!
Thank you everyone in advance.
This doesn't seem like a Qlik-related question, it seems like a case of understanding how you're recognize that a row has been deleted in the source system / database.
Depending on the specifics of the source, you might be able to pull all rows with status disabled/deleted and then remove those from your Qlik source. This isn't really an incremental load but it might be the closest thing available if you have no way of recognizing that rows have been modified in the source system. This should be a valid approach of getting all of the disabled/deleted rows is fairly quick.
Hello @Or , thank you for your reply. I understand, but I don't have rights/access to the source database and I'm asked to implement incremental load without having what I need, not even flag fields...
I was thinking maybe I could run the rest of the incremental scripts and when it comes to these tables, I could do a script that drops these tables and remakes them all over again from the start? Is that do-able?
There shouldn't be a problem using that approach, but it certainly doesn't qualify as being incremental if you're reloading the whole table. If that's not an issue, this would likely be a good option.
If you go this way, do keep in mind that if other queries/tables depend on this one (using joins/keeps/etc) you would need to deal with that as well.