Load in incremental mode with TOS for Data Integration

[INESBK]

Hi,

I read a large volume of files (which have the same format and the same columns) in order to insert them in sql server.
Each time I have to insert(load) new lines of files (each evening for example).

My question is: Can i use INSERT IF Not Exist(action in data) to do the trick or I have to create a CDC.

What is the best solution?
If the solution is CDC how can I create it?

Best regards.

Thanks in advance.

[Anonymous]

Hello,

 Talend Changed Data Capture(CDC) feature will be available in talend subscription solution.

If you want to capture the changed data and only load these changed data into target table to achieve table sync, you can compare tables by using tMap.
The work flow should be:

Target DB-->tMap(make inner join on your input and set the "Catch lookup inner join reject" as true)-->output
Source DB-->
The output will be the changed data.

Let us know if it is Ok with you.

Best regard

Sabrina

[INESBK]

Could you please explain to me more. 

My job is like this 

tfile_list----->tSystem----->tExtractDelimetedFields----->tMap----->tMsSqlOutput_1------>tMsSqlOutput_2(for reject rows)

So i should to connet tMsSqlOutput_1 with an other tMap to consider it as a source?

Thanks.