maintaining another DB for history records is good ??
Hello ,
We have 5k contracts, each contract will moved to history cycles after a year of its starting time. as of now 4k contracts are in the history cycle.
we are doing all the calculations in the DB, in order to save the time and memory in Qlik, Now the issue is all these history and latest cycles of data are into the same tables in DB, now the database team plan is every week they run the jobs related to history data so that new contracts will be added to history contracts list. and along with that our regular hourly refresh will get the updated contracts data.
now we are thinking to maintain another DB with same table structure for the history data, so that we can just run only once in a weekend and append the new contracts to the history contracts and load them into Qlik along with other latest cycles data.
feeling like its waste of time & memory if we maintain everything in a single data , as every hour it has to check for the all the records to identify the latest updates, although history data will not be updated that much frequently.
Please suggest best method to utilize our DEV and PROD servers effectively without any time lags, we have to implement this incrementals here.
maintaining a separate tables in the same DB ??
maintaining a separate DB for history data and run on weekly basis and add the qvds to latest cycles ??
