topic Incremental Load, Error Logging, and notifications? in Talend Studio

Incremental Load, Error Logging, and notifications?

Anonymous — Sat, 16 Nov 2024 04:09:31 GMT

Hi! Sorry for the many questions but I'm just learning about a lot of things that's possible except don't know how to execute the best practices.

So for my project, i'm loading data from MSSQL to Salesforce. To perform incremental loading, i created a new column in sql "talendRunTime" and every time the job is run, it will update the rows where talendRunTime is null. It will only query for those rows where talendRunTime is null too.

This will work...but encountered a small problem.

I want to only update talendRunTime column for the row that has successfully loaded data into Salesforce. The ones that gets rejected will not be updated. I have yet to figure out a way to do that.

So structure goes like this:

MSSQLINPUT (where talendRunTime is null) > tmap > Salesforce --- if success > update that row with current date | if rejected > log error and email (don't update row with date, keep null)

Any help would be greatly appreciated...thank you!!

Re: Incremental Load, Error Logging, and notifications?

manodwhb — Fri, 08 Nov 2019 05:17:12 GMT

@Zykros .you can take the Row >>Main, reject from tsalesforceoutput component and can derive the required next steps.

Re: Incremental Load, Error Logging, and notifications?

Anonymous — Fri, 08 Nov 2019 15:17:40 GMT

@manodwhb hi! i'm not quite sure what you mean...the primary question i have is if i can update rows that's been successfully processed from sql to salesforce but not the rejected ones

Thank you!

Re: Incremental Load, Error Logging, and notifications?

manodwhb — Tue, 12 Nov 2019 09:52:04 GMT

@Zykros , yes, you can capture the success ones also using Row >>Manin from the tSalesforceoutput.