I have a DB which has many tables in it. Data is inserted, updated and deleted on a daily basis. I have few QVDs setup that pulls data from these tables twice daily. Then there are many QVWs that reads data from these QVDs. My problem is that data pull from DB to QVD (complete load) is taking a long time (about 2 hours now) and its increasing with each day. Is there a way to pull only those records from DB that are inserted, modified and deleted and update the DB incrementally?
Nice presentation. For deletes, I haven't found the inner join of existing primary keys back to the table to be particularly slow. Since the primary key is typically sitting on an index, a database management system typically only needs to read that index, which can be MUCH faster than reading the table data itself. It is also reading the ENTIRE index, which a smart DBMS should be able to do in a highly-optimized way.