Qlik Community

Ask a Question

QlikView App Dev

Discussion Board for collaboration related to QlikView App Development.

Announcements
On May 18th at 10AM EDT we will answer your QlikView questions live. REGISTER
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

Apply Incremental reload to extract 1.5TB of data in few minutes..

Hi!

I want to extract approx. 1.5TB of data from database. How can i apply incremental reload to extract such a massive volume of data in few minutes? Also how i identify the change data in the database without using a date condition ? Does SQL server change data capture works here ?

Please help on this..

Thanks in advance!

3 Replies
MK_QSL
MVP
MVP

You need to have some primary key or date to use incremental load....

its_anandrjs

First time you have to do Full Data load and make a QVD then find the

1. Primary key in the data base.

2. If primary key is not available then use Composite key combination.

3. Find any updation date and compare it with last Reload time of the application.

On the basis of this points you can identify the scenario for the incremental load. I Suggesting you to find the best way is Incremental load like on the above points find the way to do only one day before updated records that is mean not load all rows daily it decreases your application reload time and save time also.

maxgro
MVP
MVP

there is no a single answer, it depends on

- dml on your db table (insert and/or update and/or delete, delete is worst case)

- field (usually date, primary key) to detect change in your table

to better understand the different possibility in the online help look for

Using qvd file for incremental load