[resolved] tSalesforceInput PK or Autonumber chunking

[JR1]

Hi all

I would like to know if the tSalesforceInput component supports PK or autonumber chunking in Bulk Query mode. Please find the details here: PK Chunking Header. This can be crucial for the performance when exporting large numbers of records.

Thanks & Regards, Joachim

[TRF]

Hi,
You may associate tSalesforceInput with a tSalesforceConnection in which "For salesforce bulk component" option is checked.
In this case, in tSalesforceInput choose "Query Mode" as "Bulk Query" (or "Mass Query" depending on TOS version) and that's all (if I understood correctly your question). 
Salesforce will process in parallel mode to retrieve the selected records (as for a bulk Insert/Update/Delete/Upsert). 
Hope this help,
TRF

[JR1]

Hi

No, unfortunately this does not answer my question and it is a bit more technical than that. I know how to tell Talend to use the Salesforce Bulk API. I cannot find the option to set the PK Chunking Header in the requests and as far as I can see this is not possible. Please see the article mentioned above to fully understand it.

Regards, Joachim

[JR1]

If anyone is interested: there is a recent Talendforge case regarding this:  https://jira.talendforge.org/browse/TDI-38096

Apparently this is not supported at the moment. It definitely should be as this would dramatically increase the performance of the Bulk API in case millions of records are to be extracted.

[TRF]

Hi,
What about using directly the Salesforce Bulk API from Talend (without using tSalesforce components)?

[JR1]

This would mean rebuilding all the functionality the tSalesforceInput component already provides for Bulk Mode in a custom Java component including building the WSC, creating the job, collecting results, error handling, etc. It would undermine the purpose of using Talend in the first place if I just use it to program my own scripts with it (this, one could do without Talend). I have no immediate need for this but tSalesforceInput must definitely support this in the near future.