Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
Not applicable

SFDC API usage questions

We upgraded our SFDC connector from version 11.0.10009.0 to version 14.1.0.10 and it looks like it blew up our API limit. I'm not a hundred percent sure that this upgrade caused all of the issue, as SFDC api usage is a little hard to track. But to try to chase it down I setup a fresh developers version of SF and ran several different connections and measured the api usage.

My results are below with the code I used to get them. Have others seen a big increase in api usage levels when going to the bulk api settings? I am wondering how often the SFDC connector, when using the bulk mode, calls SF to see if the job is complete? If you look at the 4th run in my data, while I get that the settings are not very smart for a table with 1000 records, it used 22 api calls per second to get the full table. Can anyone let me know what is going on here? We have a couple of tables with more than 10 million rows and I am worried that if they take a long time to pull that it will cause issues. 

//full table pull returns 1000 records for all pulls

//Soap API - uses 8 api calls and 0 batches (4 sec)

//CUSTOM CONNECT TO "Provider=QvSalesforceConnector.exe;mode=SOAP;soapOption=SOAPQ;timeout=100;batchSize=500;plainQueriesOnly=true;pkChunking=false;chunkSize=100000;threadCount=2;ProxyAuthentication=0;

//Bulk API - uses 17 api calls and 1 batch (6 sec)

//CUSTOM CONNECT TO "Provider=QvSalesforceConnector.exe;mode=BULK;timeout=100;batchSize=500;plainQueriesOnly=true;convertUTCToLocalTime=1;pkChunking=false;chunkSize=100000;threadCount=2;ProxyAuthentication=0;

//Bulk API with chunking  - uses 21 api calls and 2 batches (6 sec)

//CUSTOM CONNECT TO "Provider=QvSalesforceConnector.exe;mode=BULK;timeout=100;batchSize=500;plainQueriesOnly=true;pkChunking=1;chunkSize=100000;threadCount=2;ProxyAuthentication=0;

//Bulk API with chunking 10 - uses 475 api calls and 101 batches (22 sec)

//CUSTOM CONNECT TO "Provider=QvSalesforceConnector.exe;mode=BULK;timeout=100;batchSize=500;plainQueriesOnly=true;pkChunking=1;chunkSize=10;threadCount=2;ProxyAuthentication=0;

//Bulk API with chunking 100 - uses 57 api calls and 11 batches (6 sec)

//CUSTOM CONNECT TO "Provider=QvSalesforceConnector.exe;mode=BULK;timeout=100;batchSize=500;plainQueriesOnly=true;pkChunking=1;chunkSize=100;threadCount=2;ProxyAuthentication=0;

test:

LOAD *;

SELECT *

FROM Account

;

Thanks,

Jeff

0 Replies