Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

incremental load without date fields ?

Hello everyone,

I have an application in which I load data from the database without almost any transformation and then store the tables in qvd files.

I don't have any date fields like "creating data" or "modification date".  I was wondering if there was any tips to use kind of incremental load ?

Can I compare the values to the ones existing in the qvd and then add only the new lines ? But if I have to compare each record of the table to each record of the QVD, would that improve the performance ?

Thank you for your anwers

7 Replies
MK_QSL
MVP
MVP

What is the primary key in your database?

ashfaq_haseeb
Champion III
Champion III

Possible,

But atleast you need to have primary key.

Regards

ASHFAQ

Not applicable
Author

Hello,

Yes, in each table, there is an "id".

From what I've read, I can do insert but not update ?

tresesco
MVP
MVP

That's true. Without modification timestamp (and actually not comparing all field values -which beats the purpose of incremental load) updated records would not be possible to identify.

Not applicable
Author

I would like to try it but I cannot find the proper syntax:

LOAD

    1 as Compteur_Location, 

    id,

....

    ;

SQL SELECT *

FROM "MyTable";

where id not in

LOAD id from

Lala\MyTable.qvd

(qvd);

id is the primary key.

ANy ideas ?

Thank you

Not applicable
Author

I have lso tried this one :

LOAD id from

MyTable.qvd

(qvd) ;

Concatenate

LOAD

    1 as Compteur_Location, 

    id,

  ...

    where not exists(id);

SQL SELECT *

FROM MyTable

;

But again all the table is reloaded...