How to append data in Destination DB

Anonymous — Tue, 27 Aug 2019 15:04:15 GMT

Hello Everyone,

I have four jobs designed:
1) Job 1: Pull CSV files from FTP to Local Folder.
2) Job 2: Load CSV files from Local folder to MySQL source DB.
3) Job 3: Load dimension tables in destination DB from source DB.
4) Job 4: Load Fact Tables in destination DB from source DB.

After the initial data dump, I have the process set up to run once every month to receive CSV files regarding stats for each month. After Job 2, I have a truncate command for all the dimension tables and fact tables to reload the tables( so that they have the new data as well).

Instead of truncating I would like to append the new data received to the destination DB tables. Is there any way to do this?

Looking forward to your inputs.

Thanks and Regards,
Praneeth Reddy Varimalla

Re: How to append data in Destination DB

TRF — Wed, 28 Aug 2019 08:06:44 GMT

If records for dimension and fact tables never exist in the destination DB, Insert statement should be enough.

Else, if you have to update existing rows and insert new ones, have a look to Merge statement.

topic How to append data in Destination DB in Talend Studio

How to append data in Destination DB

Re: How to append data in Destination DB