Qlik Community

Qlik Sense Data Connectivity

Discussion board where members can learn more about Qlik Sense Data Connectivity.

Highlighted
brandonblaine
New Contributor III

Data Load from SQL; QvOdbcConnectorPackage.exe.config Settings

Hi,

Currently, when I load data from SQL into a Qlik Sense QVD, the Data Load Progress window shows that about 3,000 rows at a time are loaded into the QVD.

Can this be increased? The QVD is super fast obviously because it is all in memory. Loading the QVD is pretty painful though. Is there an equivalent to the SSIS fast load option?

I browsed the QvOdbcConnectorPackage.exe.config file and nothing stood out except maybe:

        <setting name="MaximumFileSize" serializeAs="String">
        <value>10MB</value> 

Here is the load script. Nothing fancy.

     Charges:

     Load *;

     SQL SELECT * FROM dbo.CHARGES;

    Store Charges into [lib://FileSystem/Charges.qvd] qvd;

    Drop table Charges;

Brandon Blaine
3 Replies
Partner
Partner

Re: Data Load from SQL; QvOdbcConnectorPackage.exe.config Settings

Weird behaviour,

 

Are you able to only load all records to memory (removing the store)?

Any error messages appear when storing to the QVD?

 

Dumb question, are there more than 3000 records on the table?

It should do quite well in loading data from SQL, i've done a few millions quite fast and storing without a problem, can you specify the exact actions are made and what goes wrong with a printscreen?

brandonblaine
New Contributor III

Re: Data Load from SQL; QvOdbcConnectorPackage.exe.config Settings

Hi felipedl,



Thanks for the reply. This seems to be the default regardless of if I am loading into a QVD or straight into the app from the SQL source. The table I have is ~2 million rows. It is only one table which is the result of a de-normalized query which I do because I don't believe Qlik handles complex data models very well.



I'd forgo the QVD step if I could but the data manager will not process that much data without it being in a QVD.




Brandon Blaine
brandonblaine
New Contributor III

Re: Data Load from SQL; QvOdbcConnectorPackage.exe.config Settings

I just had the IT team increase the number of cores on the VM from 4 to 8. I'll have to check the logs to see if this helps. The VM has 60GB of RAM.

Brandon Blaine