topic Re: Difference between Sales Force OutPut Components in Talend Studio

Difference between Sales Force OutPut Components

Anonymous — Sat, 16 Nov 2024 08:39:44 GMT

Hi,

I am trying to learn SalesForce Integration. Can anyone please tell me difference between:-

tSalesForceOutPut
tSalesForceOutPutBulkExec

Regards

Tarunjit Singh

Re: Difference between Sales Force OutPut Components

manodwhb — Tue, 20 Feb 2018 10:46:39 GMT

The Generic out put component will do inert/upsert/update/delete operation on specific SF table,even with Bulk one also similar but the Bulk one first it will cretae a Bulk file and then it will try to the operations based on the Bulk file.

https://help.talend.com/reader/iYcvdknuprDzYycT3WRU8w/DdNFQ29mYUxqVhLa8fOMQw

Re: Difference between Sales Force OutPut Components

Anonymous — Tue, 20 Feb 2018 10:52:35 GMT

Hi,

But why would we want bulk file to be first created on server and then upload that bulk file content to SalesForce. Is there some specific advantage of creating this bulk file?

Regards

Tarunjit Singh

Re: Difference between Sales Force OutPut Components

TRF — Tue, 20 Feb 2018 10:53:40 GMT

The main difference is that they do not use the Salesforce API - always consider the Bulk API as soon as you have a huge data volume or want to deal easilly with rejected records.

Re: Difference between Sales Force OutPut Components

manodwhb — Tue, 20 Feb 2018 10:54:18 GMT

in the normal out it will process record by record,where from Bulk file it will do Bulk operation,Bulk one would be faster.

Re: Difference between Sales Force OutPut Components

TRF — Tue, 20 Feb 2018 11:06:13 GMT

tSalesforceOutput doesn't strickly work row by row, it has a "Commit Level" which fixe the transaction size.

As soon as 1 row is rejected due to a internal Salesforce reason, the whole transaction is lost.

With Bulk API, if a record is rejected the others are nore affected.

Re: Difference between Sales Force OutPut Components

manodwhb — Tue, 20 Feb 2018 11:08:42 GMT

based on the commit level it will insrt row by row. correct that where in batch ,if any bad records is there means entire batch willl reject.

Re: Difference between Sales Force OutPut Components

Anonymous — Tue, 20 Feb 2018 12:14:06 GMT

Hi,

Did you mean if 5 records have to be inserted using tSalesForceOutPutBulkExec and out of these 5 records 1 record get some error then 4 records will gets inserted and 1 record will be rejected.

However if 5 records have to be inserted using tSalesForceOutPut and out of these 5 records 1 record get then all of the 5 records will get rejected.

Regards

Tarunjit Singh

Re: Difference between Sales Force OutPut Components

manodwhb — Tue, 20 Feb 2018 12:20:11 GMT

in Bulk all the records rejected if 1 record having error means,

in output,it will insert 4 records and 1 record will be rejected where in same scenario.

Re: Difference between Sales Force OutPut Components

Anonymous — Tue, 20 Feb 2018 12:41:55 GMT

Hi,

I just tried it what you said however it is behaving in opposite manner.

When I tried to add 5 records in Sales Force using tSalesForceOutPut and 1 record have some error then no record gets added infact we got run time error as mentioned below. Also job didn't even completed its full flow that is flow doesn't enter in to Reject or Main flow.

Error Message

Exception in component tSalesforceOutput_1 (testSalesForceBulkJob)

java.io.IOException: Required fields are missing: [Name]

However when I tried to add 5 records in Sales Force using tSalesForceOutPutBulkExec and 1 record have some error then 4 records got added and 1 record not got added. More over there is no error when the job is run as is mentioned in tSalesForceOutPut scenario. In this scenario job completed its flow completely that is flow enter successfully in to Reject or Main flow.

Re: Difference between Sales Force OutPut Components

TRF — Tue, 20 Feb 2018 13:05:54 GMT

Untick the option "Cease on Error" if you din't want the job to stop in case of error.

You may also refer to this post where I tried to explain how it works.

Re: Difference between Sales Force OutPut Components

Anonymous — Tue, 20 Feb 2018 13:22:23 GMT

Hi,

Thanks for the answer. Job is not now not breaking but no response from Sales Force is coming, i.e in main flow and reject flow it is showing 0 rows.

It seems that if we use tSalesForceOutPutBulkExec then if a record is rejected i.e. got failed while trying to add then others records are not effected. e.g. 1 out of 5 records are rejected then remaining 4 will get added in to Sales Force.

Where as if we use tSalesForceOutPut then if a record is rejected i.e. got failed while trying to add then others records will also be effected. e.g. 1 out of 5 records are rejected then remaining 4 will also not get added in to Sales Force and also there will also be no response from Sales Force regarding success and failure as mentioned in my first paragraph.

Re: Difference between Sales Force OutPut Components

Anonymous — Tue, 20 Feb 2018 13:28:04 GMT

Hi,

I think I resolved the problem by setting Commit Level To1. Now tSalesForceOutPut is able to add 4 success records in to Sales Force and also I got response for failure.

But I think it will be slow as Commit Level is set to 1. As transaction will be created for each record. Please confirm.

Re: Difference between Sales Force OutPut Components

Anonymous — Thu, 22 Feb 2018 11:18:42 GMT

Hi,

Here is my understanding as per your comment.

Suppose there are 50 records being added from tSalesForceOutPutBulkExec and few of the records got failed due to sales force internal error. In that scenario whole of the batch will get failed.
Suppose there are 50 records being added from tSalesForceOutPut and few of the records got failed due to sales force internal error. In that scenario not whole of the batch will get failed only those of the records will get failed.

Please let me know if my understanding is correct.

Re: Difference between Sales Force OutPut Components

manodwhb — Thu, 22 Feb 2018 11:21:02 GMT

Yes correct..

Re: Difference between Sales Force OutPut Components

Anonymous — Thu, 22 Feb 2018 11:46:05 GMT

Hi,

So in case of bulk addition using tSalesForceOutPutBulkExec if there are 100 records and batch size of 50 and first batch got failed due to Sales Force Internal issue then on response I will receive 50 records for success flow and 50 records for reject flow.

Please confirm.

Re: Difference between Sales Force OutPut Components

manodwhb — Thu, 22 Feb 2018 12:22:59 GMT

if the any one batch has issue then all the records wil be failed in BULK mode.

Re: Difference between Sales Force OutPut Components

Anonymous — Thu, 22 Feb 2018 12:34:02 GMT

Hi,

It means if one batch fails then all the records of all the batches will have the same error message received from sales force.

Re: Difference between Sales Force OutPut Components

manodwhb — Thu, 22 Feb 2018 12:37:03 GMT

you will get below kind of error.

Exception in component tSalesforceOutputBulkExec_2_tSalesforceBulkExec
java.io.IOException: [AsyncApiException exceptionCode='InvalidBatch'
exceptionMessage='Records not processed'
]

at org.talend.components.salesforce.runtime.SalesforceBulkExecReader.start(SalesforceBulkExecReader.java:76)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tSalesforceInput_3Process(DataMig_OpportunityItem_Unify.java:21166)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tHashInput_10Process(DataMig_OpportunityItem_Unify.java:18922)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tHashInput_1Process(DataMig_OpportunityItem_Unify.java:14040)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tHashInput_18Process(DataMig_OpportunityItem_Unify.java:10458)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tFileList_2Process(DataMig_OpportunityItem_Unify.java:4397)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tSalesforceConnection_3Process(DataMig_OpportunityItem_Unify.java:3942)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tSalesforceConnection_1Process(DataMig_OpportunityItem_Unify.java:3713)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tLoop_1Process(DataMig_OpportunityItem_Unify.java:3416)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tJava_3Process(DataMig_OpportunityItem_Unify.java:2781)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tFileList_1Process(DataMig_OpportunityItem_Unify.java:1923)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tJava_15Process(DataMig_OpportunityItem_Unify.java:23526)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tFTPGet_1Process(DataMig_OpportunityItem_Unify.java:23358)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tFTPConnection_1Process(DataMig_OpportunityItem_Unify.java:23019)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tJava_12Process(DataMig_OpportunityItem_Unify.java:22781)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tJava_9Process(DataMig_OpportunityItem_Unify.java:22613)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tFixedFlowInput_2Process(DataMig_OpportunityItem_Unify.java:24047)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tJava_10Process(DataMig_OpportunityItem_Unify.java:24299)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.tSetProxy_1Process(DataMig_OpportunityItem_Unify.java:37203)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.runJobInTOS(DataMig_OpportunityItem_Unify.java:37606)
at atos.datamig_opportunityitem_unify_1_0.DataMig_OpportunityItem_Unify.main(DataMig_OpportunityItem_Unify.java:37298)
Caused by: [AsyncApiException exceptionCode='InvalidBatch'
exceptionMessage='Records not processed'

Re: Difference between Sales Force OutPut Components

Anonymous — Thu, 22 Feb 2018 12:47:10 GMT

Hi,

From the error message it seems that in such case when a batch will fail then the complete job will get fail. That is flow will not move forward from tSalesForOutPutBulkExec. Please confirm.