Is tSalesforceOutput thread safe?

Anonymous — Thu, 16 May 2019 19:34:19 GMT

I am trying to streamline the loading of ContentVersion records (attachments) to salesforce by using a tFlowtoIterate with enable parallel execution to parallelize a call to a child job. I have also tried this without a child job and got the same result.

ContentVersions have to be loaded serially into salesforce, they can't be grouped in to batches greater than 1 record owing to various restrictions in salesforce.

So I am trying to work around this by setting up many threads all running a single record into salesforce.

To start with the job runs very fast with multiple threads executing to load into salesforce. However after n records the tSalesforceOutput component fails with the following string index error.

This error does not occur if I set the tRunJob to use separate processes for each execution however the overhead of setting up a new process for each iteration slows the process down so much it is no faster than the serial method.

So I suspect that something in tSalesforceOutput is not thread safe.

Has anyone seen this behaviour before or know if my hunch is correct?

----

Exception in component tRunJob_1 (PutAttachmentParent)
java.lang.RuntimeException: Child job running failed.
java.lang.StringIndexOutOfBoundsException: String index out of range: -8
at ihs.putattachmentparent_0_3.PutAttachmentParent$1tRunJob_1Thread.run(PutAttachmentParent.java:4970)
at routines.system.ThreadPoolWorker.runIt(TalendThreadPool.java:159)
at routines.system.ThreadPoolWorker.runWork(TalendThreadPool.java:150)
at routines.system.ThreadPoolWorker.access$0(TalendThreadPool.java:145)
at routines.system.ThreadPoolWorker$1.run(TalendThreadPool.java:122)
at java.lang.Thread.run(Unknown Source)

----Here is the exception log if I don't use a child job

18:30:37,049 INFO [SalesforceWriter] Done.
18:31:26,508 INFO [SalesforceWriter] Prepare to insert record 1.
18:31:27,250 INFO [SalesforceWriter] Has inserted record 1 successfully.
18:31:27,250 INFO [SalesforceWriter] Has inserted 1 records successfully.
18:31:27,250 INFO [SalesforceWriter] Has rejected 0 records.
18:31:27,250 INFO [SalesforceWriter] Done.
18:31:27,828 INFO [SalesforceWriter] Failed to insert record 1.
18:31:27,828 INFO [SalesforceWriter] Has inserted 1 records successfully.
18:31:27,828 INFO [SalesforceWriter] Has rejected 1 records.
18:31:27,828 INFO [SalesforceWriter] Done.
Exception in component tSalesforceOutput_1 (PutAttachmentParallel)
java.lang.StringIndexOutOfBoundsException: String index out of range: 13
at java.lang.String.substring(Unknown Source)
at java.text.MessageFormat.subformat(Unknown Source)
at java.text.MessageFormat.format(Unknown Source)
at java.text.Format.format(Unknown Source)
at org.talend.daikon.i18n.I18nMessages.getFormattedMessage(I18nMessages.java:63)
at org.talend.daikon.i18n.ClassBasedI18nMessages.getMessage(ClassBasedI18nMessages.java:76)
at org.talend.components.salesforce.runtime.SalesforceWriter.getPastForm(SalesforceWriter.java:738)
at org.talend.components.salesforce.runtime.SalesforceWriter.handleSuccess(SalesforceWriter.java:511)
at org.talend.components.salesforce.runtime.SalesforceWriter.doInsert(SalesforceWriter.java:351)
at org.talend.components.salesforce.runtime.SalesforceWriter.insert(SalesforceWriter.java:328)
at org.talend.components.salesforce.runtime.SalesforceWriter.write(SalesforceWriter.java:185)
at ihs.putattachmentparallel_0_3.PutAttachmentParallel$1tDBInput_2Thread.run(PutAttachmentParallel.java:8802)
at routines.system.ThreadPoolWorker.runIt(TalendThreadPool.java:159)
at routines.system.ThreadPoolWorker.runWork(TalendThreadPool.java:150)
at routines.system.ThreadPoolWorker.access$0(TalendThreadPool.java:145)
at routines.system.ThreadPoolWorker$1.run(TalendThreadPool.java:122)
at java.lang.Thread.run(Unknown Source)

Re: Is tSalesforceOutput thread safe?

Anonymous — Tue, 21 May 2019 09:02:15 GMT

Hello,

The Salesforce output component actually works better with a tFlowToIterate component placed in front. Could you please show us your current job design screenshots? Did you use global variables(Put, Get) in your work flow?

Best regards

Sabrina

Re: Is tSalesforceOutput thread safe?

Anonymous — Tue, 21 May 2019 11:13:20 GMT

Hi,

Yes I tried the tFlowToIterate pattern with the same result.

In this example I've tried running 4 in parallel but by splitting the flow using a randomly assigned number. when I set the enable parallel execution on the iterate links and set the number to >1 I get the error mentioned in my post.

The tMaps doing the lookup to salesforce to get the attachment data are operating in the reload each row mode to query for a specific salesforce record each time.

PutAttachmentParallelTest_0.3.png

topic Re: Is tSalesforceOutput thread safe? in Talend Studio

Is tSalesforceOutput thread safe?

Re: Is tSalesforceOutput thread safe?

Re: Is tSalesforceOutput thread safe?