Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi,
I am trying to query data from Salesforce using the standard tSalesforceInput component together with a previous tSalesforceConnection with bulk enabled.
The query should retrieve about 1M data, so I have already set the bulk mode with a large timeout on the connection component, but when I run it, after 2 minutes exactly it gets timed out, while if I run it from Salesforce workbench, after 20 mins the results are retrieved without time outs.
Is there any other way to avoid the time out?
Thanks
I'm afraid you have reach the timeout limit for a Bulk query.
As 20 minutes is very very long for just 1 million records, you should try to simply your query or review the job design.
Sometimes, having 2 simple queries is better than 1 complex query.
Do you mean timeout for bulk query via Talend components? Because if I run the same query with other tools (workbench), I get the results after 20 mins, but with Talend I get the time out after just 2 minutes.
It looks like the time out I set in the tSalesorceConnection is not considered
I think tSalesforceInput component doesn't implement the retry strategy
https://developer.salesforce.com/docs/#QueryLimits-7C7FC75C
Can you get some information regarding your jobs when you go to the Salesforce "Bulk Data Load Jobs" setup page?
If i look the Bulk Data Load job page, I see that in some cases the query returns the 1M records but it stays in progress, so I even need to abort it manually.
I have found an interesting workaround for this issue. I was running a SF query that worked sometimes and timed out at other times. In fact, if I ran it straight away after it had initially timed out, it would generally work. Normally a more precise query will help with this or a smaller set of data. But sometimes you really can't reduce the data you need.
As a workaround for this, I have started using a tRunJob as a wrapper for my SF query component. I simply create a job with my query component and a tBufferOutput component to return my records. By doing this I can rerun the child job immediately using a tLoop IF it fails. This stops the job crashing and allows me to get around this quirky issue of some SF queries failing on a first attempt.
Below if a screenshot of how I have configured the tRunJob in my main job to replace the tSalesforceInput.....
Maybe something worth trying @Shicong Hong, @Xiaodi Shi and @Fred Trebuchet