topic tSalesforceInput - Bug with Query Mode=Bulk Query? in Talend Studio

tSalesforceInput - Bug with Query Mode=Bulk Query?

Anonymous — Sat, 16 Nov 2024 11:54:43 GMT

Has anyone any experience with using tSalesforceInput in Bulk Query mode (TOS 5.2.2)?
Getting the following error: -
Exception in component tSalesforceInput_1
com.sforce.async.CSVReader$CSVParseException: Exceeded number of records : 10002. Number of records should be less than or equal to 10001
at com.sforce.async.CSVReader.checkRecordExceptions(CSVReader.java:107)
at com.sforce.async.CSVReader.nextRecord(CSVReader.java:139)
at org.talend.salesforceBulk.SalesforceBulkAPI.getQueryResult(SalesforceBulkAPI.java:495)
at
There's an old closed JIRA against tSalesforceBulkExec https://jira.talendforge.org/browse/TDI-7779, and, from a Salesforce comment: -
"The issue is reusing com.sforce.async.CSVReader to parse the csv response from salesforce. When uploading more that 10k records it returns more than 10k results. SalesforceBulkAPI.getBatchLog and SalesforceBulkAPI.getFileRow re-use com.sforce.async.CSVReader, which blows up when parsing more than 10k rows. The solution is to not re-use com.sforce.async.CSVReader when parsing the response."
Has any one used this mode successfully? Is this a bug in Talend with its implementation of the underlying API?

Re: tSalesforceInput - Bug with Query Mode=Bulk Query?

Anonymous — Mon, 09 Sep 2013 09:36:11 GMT

This does appear to be a bug...
https://community.talend.com/t5/Design-and-Development/resolved-Salesforce-Bulk-Queries-for-Large-data-sets/td-p/83328

Re: tSalesforceInput - Bug with Query Mode=Bulk Query?

Anonymous — Mon, 09 Sep 2013 09:40:58 GMT

Hi,
It is welcome to open a new bug for this issue. Our component developer will check it.
Best regards
Sabrina

Re: tSalesforceInput - Bug with Query Mode=Bulk Query?

Anonymous — Tue, 23 Aug 2016 01:35:01 GMT

Having the same problem after upgrading from 6.1 to 6.2. Seems and old bug has been re-introduced. Anybody have a workaround or idea on when this might be fixed? Help would be much appreciated as we have many jobs utilising Bulk Query mode. If I have to revert to Query mode the impact is extended processing time and increase in API call count.
Thanks in advance, Grant
Exception in component tSalesforceInput_1
com.sforce.async.CSVReader$CSVParseException: Exceeded number of records : 10002. Number of records should be less than or equal to 10001
at com.sforce.async.CSVReader.checkRecordExceptions(CSVReader.java:159)
at com.sforce.async.CSVReader.nextRecord(CSVReader.java:189)
at org.talend.components.salesforce.runtime.SalesforceBulkRuntime$BulkResultSet.next(SalesforceBulkRuntime.java:593)
at org.talend.components.salesforce.runtime.SalesforceBulkQueryInputReader.advance(SalesforceBulkQueryInputReader.java:71)
at local_project.asset_all_0_1.Asset_All.tSalesforceInput_1Process(Asset_All.java:1748)
at local_project.asset_all_0_1.Asset_All.runJobInTOS(Asset_All.java:2527)
at local_project.asset_all_0_1.Asset_All.main(Asset_All.java:2384)

Re: tSalesforceInput - Bug with Query Mode=Bulk Query?

Anonymous — Fri, 02 Dec 2016 15:01:43 GMT

I am having the same problem Studio 6.2.1.20160704 trying to query 1.1 million account records. Have you opened a case or found a solution? If not, I will open a case as well. Thanks.

Re: tSalesforceInput - Bug with Query Mode=Bulk Query?

Anonymous — Mon, 19 Dec 2016 08:51:37 GMT

Edit : FYI, After opening a case, Talend support revert with a patch to fix the issue. Hope our feedback helps.
Hi all,
Having the same error message, trying to run a job with Talend Studio 6.2.1. any update on this case ?
On our side, a case was opened too.
Best regards,