Logic Check on Repository Layer for Incremental ADD

schneider2037 — Wed, 25 Nov 2020 16:16:04 GMT

I have an Issues and Issues Call (think service calls) that have a unique ISSUE_ID that links the Issues and Issues Calls together. For the sake of this question, will stick with just Issues, as the Issues Calls will operate exactly the same (I HOPE).

The daily file contains any new Issues (based on a new, unique ISSUE_ID), and need to merge that with the prior data. It's possible that an existing ID might have some information updated, in which case the entire row is resent along with any "new" ISSUE_ID.

Question here is that I want to create a backup of the "historical" data first before firing in new data. My concern is that I'm loading ServiceISSUEHistory first, turning around and firing out a .qvd as a differently named backup, then dropping it so I'm always starting with the "most current state" of any given ISSUE_ID (be it new or any updated ones).

1 LOAD ServiceISSUEHistory.qvd as table "ServiceISSUEHistory:"

2 STORE as ServiceISSUEHistoryBackup.qvd {in case something goes horribly wrong have a prior record somewhere}

3 DROP table ServiceISSUEHistory {need to start with the most current version of any ISSUE_ID)

4 LOAD new file Service_Issue_Past7days_*.xls {daily file comes over as .xls)

5 LOAD ServiceISSUEHistory.qvd WHERE NOT EXISTS (ISSUE_ID) {only bring it in if it DOESN'T already exist}

6 STORE as ServiceISSUEHistory.qvd {overwrite the existing historical record with today's rows added in}

7 Move on to other tables to load into the data model.

Is this a sound series of steps to do this? Or am I barking mad for trying it this way?

Thanks for any and all opinions on the matter!

Jonathan

topic Logic Check on Repository Layer for Incremental ADD in QlikView

Logic Check on Repository Layer for Incremental ADD