0 Replies Latest reply: Nov 30, 2016 1:08 AM by Jeff Penner RSS

    SFDC API usage questions

    Jeff Penner

      We upgraded our SFDC connector from version 11.0.10009.0 to version 14.1.0.10 and it looks like it blew up our API limit. I'm not a hundred percent sure that this upgrade caused all of the issue, as SFDC api usage is a little hard to track. But to try to chase it down I setup a fresh developers version of SF and ran several different connections and measured the api usage.

       

      My results are below with the code I used to get them. Have others seen a big increase in api usage levels when going to the bulk api settings? I am wondering how often the SFDC connector, when using the bulk mode, calls SF to see if the job is complete? If you look at the 4th run in my data, while I get that the settings are not very smart for a table with 1000 records, it used 22 api calls per second to get the full table. Can anyone let me know what is going on here? We have a couple of tables with more than 10 million rows and I am worried that if they take a long time to pull that it will cause issues. 

       

      //full table pull returns 1000 records for all pulls

       

      //Soap API - uses 8 api calls and 0 batches (4 sec)

      //CUSTOM CONNECT TO "Provider=QvSalesforceConnector.exe;mode=SOAP;soapOption=SOAPQ;timeout=100;batchSize=500;plainQueriesOnly=true;pkChunking=false;chunkSize=100000;threadCount=2;ProxyAuthentication=0;

       

      //Bulk API - uses 17 api calls and 1 batch (6 sec)

      //CUSTOM CONNECT TO "Provider=QvSalesforceConnector.exe;mode=BULK;timeout=100;batchSize=500;plainQueriesOnly=true;convertUTCToLocalTime=1;pkChunking=false;chunkSize=100000;threadCount=2;ProxyAuthentication=0;

       

      //Bulk API with chunking  - uses 21 api calls and 2 batches (6 sec)

      //CUSTOM CONNECT TO "Provider=QvSalesforceConnector.exe;mode=BULK;timeout=100;batchSize=500;plainQueriesOnly=true;pkChunking=1;chunkSize=100000;threadCount=2;ProxyAuthentication=0;

       

      //Bulk API with chunking 10 - uses 475 api calls and 101 batches (22 sec)

      //CUSTOM CONNECT TO "Provider=QvSalesforceConnector.exe;mode=BULK;timeout=100;batchSize=500;plainQueriesOnly=true;pkChunking=1;chunkSize=10;threadCount=2;ProxyAuthentication=0;

       

      //Bulk API with chunking 100 - uses 57 api calls and 11 batches (6 sec)

      //CUSTOM CONNECT TO "Provider=QvSalesforceConnector.exe;mode=BULK;timeout=100;batchSize=500;plainQueriesOnly=true;pkChunking=1;chunkSize=100;threadCount=2;ProxyAuthentication=0;

       

      test:

      LOAD *;

      SELECT *

      FROM Account

      ;

       

      Thanks,

      Jeff