topic tSalesforceInput - Query Mode: Bulk Exception in Talend Studio

tSalesforceInput - Query Mode: Bulk Exception

jdope — Sat, 16 Nov 2024 08:38:23 GMT

I am using a tSalesforceInput to retrieve data using the Bulk query mode. I am able to create the bulk data load job in salesforce successfully, see it execute and retrieve the results as long as the bulk job completes in a "quick time-frame".

I have the same job, salesforce environment, etc and the only thing I change is the query parameters - to return more data. This query takes more time and it seems in that "waiting" that some error is thrown by Talend. In reality the job completes (takes 3 minutes on salesforce) and the data is available in the salesforce job. For some reason Talend is deciding to "time-out" on the bulk data load job. This is not a network issue - I can repeat it consistently.

I should also mention that the Timeout value in the tSalesforceConnection component is set to 480000 (8 minutes).

Attached is the exception from Talend.

Successful Salesforce Job; Same job that Talend has an exception on:

Re: tSalesforceInput - Query Mode: Bulk Exception

TRF — Sat, 24 Feb 2018 18:40:02 GMT

Hi,
On Salesforce side there is some limits regarding the Bulk API including for queries. As explained in the documentation, "If the query takes more than 2 minutes to process, a QUERY_TIMEOUT error is returned".
Here is the link https://developer.salesforce.com/docs/atlas.en-us.api_asynch.meta/api_asynch/asynch_api_concepts_limits.htm

Re: tSalesforceInput - Query Mode: Bulk Exception

jdope — Sat, 24 Feb 2018 20:06:10 GMT

Thanks for the reply.

I put a HTTP trace on the connection to see the responses from Talend polling the bulk job in salesforce. It shows the responses from each poll to salesforce to get the progress of the job (see the attached text file). I see nothing indicating that the job returned any such error condition. In salesforce the batch completes successfully and the results are available (see screenshot in post).

I'm not sure what Talend is looking at in the response to indicate that there was an error...there doesn't seem to be one returned by salesforce. Talend doesn't seem to be processing the response correctly and throws an exception (why? I don't know).

Re: tSalesforceInput - Query Mode: Bulk Exception

jdope — Thu, 01 Mar 2018 03:07:40 GMT

Created Bug

https://jira.talendforge.org/browse/TDI-40149.

If you encounter this same issue please vote up the bug!