topic tSalesforceInput query timeout in Talend Studio

tSalesforceInput query timeout

Anonymous — Sat, 16 Nov 2024 07:24:18 GMT

Hi,

I am trying to query data from Salesforce using the standard tSalesforceInput component together with a previous tSalesforceConnection with bulk enabled.

The query should retrieve about 1M data, so I have already set the bulk mode with a large timeout on the connection component, but when I run it, after 2 minutes exactly it gets timed out, while if I run it from Salesforce workbench, after 20 mins the results are retrieved without time outs.

Is there any other way to avoid the time out?

Thanks

Re: tSalesforceInput query timeout

TRF — Thu, 25 Oct 2018 12:54:59 GMT

I'm afraid you have reach the timeout limit for a Bulk query.

As 20 minutes is very very long for just 1 million records, you should try to simply your query or review the job design.

Sometimes, having 2 simple queries is better than 1 complex query.

Re: tSalesforceInput query timeout

Anonymous — Thu, 25 Oct 2018 14:26:48 GMT

Do you mean timeout for bulk query via Talend components? Because if I run the same query with other tools (workbench), I get the results after 20 mins, but with Talend I get the time out after just 2 minutes.

It looks like the time out I set in the tSalesorceConnection is not considered

Re: tSalesforceInput query timeout

TRF — Thu, 25 Oct 2018 14:39:44 GMT

I think tSalesforceInput component doesn't implement the retry strategy

https://developer.salesforce.com/docs/#QueryLimits-7C7FC75C

Can you get some information regarding your jobs when you go to the Salesforce "Bulk Data Load Jobs" setup page?

Re: tSalesforceInput query timeout

Anonymous — Mon, 29 Oct 2018 16:54:53 GMT

If i look the Bulk Data Load job page, I see that in some cases the query returns the 1M records but it stays in progress, so I even need to abort it manually.

Re: tSalesforceInput query timeout

Alinea — Thu, 28 Mar 2019 13:22:32 GMT

Hi,
we have exactly the same problem here.

Any help?

Re: tSalesforceInput query timeout

Anonymous — Wed, 03 Feb 2021 16:34:44 GMT

I have found an interesting workaround for this issue. I was running a SF query that worked sometimes and timed out at other times. In fact, if I ran it straight away after it had initially timed out, it would generally work. Normally a more precise query will help with this or a smaller set of data. But sometimes you really can't reduce the data you need.

As a workaround for this, I have started using a tRunJob as a wrapper for my SF query component. I simply create a job with my query component and a tBufferOutput component to return my records. By doing this I can rerun the child job immediately using a tLoop IF it fails. This stops the job crashing and allows me to get around this quirky issue of some SF queries failing on a first attempt.

Below if a screenshot of how I have configured the tRunJob in my main job to replace the tSalesforceInput.....

Maybe something worth trying @Shicong Hong, @Xiaodi Shi and @Fred Trebuchet