Apply Incremental reload to extract 1.5TB of data in few minutes..
I want to extract approx. 1.5TB of data from database. How can i apply incremental reload to extract such a massive volume of data in few minutes? Also how i identify the change data in the database without using a date condition ? Does SQL server change data capture works here ?
First time you have to do Full Data load and make a QVD then find the
1. Primary key in the data base.
2. If primary key is not available then use Composite key combination.
3. Find any updation date and compare it with last Reload time of the application.
On the basis of this points you can identify the scenario for the incremental load. I Suggesting you to find the best way is Incremental load like on the above points find the way to do only one day before updated records that is mean not load all rows daily it decreases your application reload time and save time also.