tSalesforceInput query timeout

[Anonymous]

Hi,

I am trying to query data from Salesforce using the standard tSalesforceInput component together with a previous tSalesforceConnection with bulk enabled. 

The query should retrieve about 1M data, so I have already set the bulk mode with a large timeout on the connection component, but when I run it, after 2 minutes exactly it gets timed out, while if I run it from Salesforce workbench, after 20 mins the results are retrieved without time outs.

Is there any other way to avoid the time out?

Thanks

[TRF]

I'm afraid you have reach the timeout limit for a Bulk query.

As 20 minutes is very very long for just 1 million records, you should try to simply your query or review the job design.

Sometimes, having 2 simple queries is better than 1 complex query.

[Anonymous]

Do you mean timeout for bulk query via Talend components? Because if I run the same query with other tools (workbench), I get the results after 20 mins, but with Talend I get the time out after just 2 minutes.

It looks like the time out I set in the tSalesorceConnection is not considered

[TRF]

I think tSalesforceInput component doesn't implement the retry strategy 

https://developer.salesforce.com/docs/#QueryLimits-7C7FC75C

Can you get some information regarding your jobs when you go to the Salesforce "Bulk Data Load Jobs" setup page?

[Anonymous]

If i look the Bulk Data Load job page, I see that in some cases the query returns the 1M records but it stays in progress, so I even need to abort it manually.

[Alinea]

Hi,
we have exactly the same problem here.

Any help?

[Anonymous]

I have found an interesting workaround for this issue. I was running a SF query that worked sometimes and timed out at other times. In fact, if I ran it straight away after it had initially timed out, it would generally work. Normally a more precise query will help with this or a smaller set of data. But sometimes you really can't reduce the data you need.

As a workaround for this, I have started using a tRunJob as a wrapper for my SF query component. I simply create a job with my query component and a tBufferOutput component to return my records. By doing this I can rerun the child job immediately using a tLoop IF it fails. This stops the job crashing and allows me to get around this quirky issue of some SF queries failing on a first attempt.

Below if a screenshot of how I have configured the tRunJob in my main job to replace the tSalesforceInput.....

 

Maybe something worth trying @Shicong Hong​, @Xiaodi Shi​ and @Fred Trebuchet​