Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
brandonblaine
Contributor III
Contributor III

Data Load from SQL; QvOdbcConnectorPackage.exe.config Settings

Hi,

Currently, when I load data from SQL into a Qlik Sense QVD, the Data Load Progress window shows that about 3,000 rows at a time are loaded into the QVD.

Can this be increased? The QVD is super fast obviously because it is all in memory. Loading the QVD is pretty painful though. Is there an equivalent to the SSIS fast load option?

I browsed the QvOdbcConnectorPackage.exe.config file and nothing stood out except maybe:

        <setting name="MaximumFileSize" serializeAs="String">
        <value>10MB</value> 

Here is the load script. Nothing fancy.

     Charges:

     Load *;

     SQL SELECT * FROM dbo.CHARGES;

    Store Charges into [lib://FileSystem/Charges.qvd] qvd;

    Drop table Charges;

Brandon Blaine
Labels (2)
3 Replies
felipedl
Partner - Specialist III
Partner - Specialist III

Weird behaviour,

 

Are you able to only load all records to memory (removing the store)?

Any error messages appear when storing to the QVD?

 

Dumb question, are there more than 3000 records on the table?

It should do quite well in loading data from SQL, i've done a few millions quite fast and storing without a problem, can you specify the exact actions are made and what goes wrong with a printscreen?

brandonblaine
Contributor III
Contributor III
Author

Hi felipedl,



Thanks for the reply. This seems to be the default regardless of if I am loading into a QVD or straight into the app from the SQL source. The table I have is ~2 million rows. It is only one table which is the result of a de-normalized query which I do because I don't believe Qlik handles complex data models very well.



I'd forgo the QVD step if I could but the data manager will not process that much data without it being in a QVD.




Brandon Blaine
brandonblaine
Contributor III
Contributor III
Author

I just had the IT team increase the number of cores on the VM from 4 to 8. I'll have to check the logs to see if this helps. The VM has 60GB of RAM.

Brandon Blaine