Qlik Community

Qlik Sense Data Connectivity

Discussion board where members can learn more about Qlik Sense Data Connectivity.

Announcements
BI & Data Trends 2021. Discover the top 10 trends emerging in today. Join us on Dec. 8th REGISTER
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Contributor III
Contributor III

Data Load from SQL; QvOdbcConnectorPackage.exe.config Settings

Hi,

Currently, when I load data from SQL into a Qlik Sense QVD, the Data Load Progress window shows that about 3,000 rows at a time are loaded into the QVD.

Can this be increased? The QVD is super fast obviously because it is all in memory. Loading the QVD is pretty painful though. Is there an equivalent to the SSIS fast load option?

I browsed the QvOdbcConnectorPackage.exe.config file and nothing stood out except maybe:

        <setting name="MaximumFileSize" serializeAs="String">
        <value>10MB</value> 

Here is the load script. Nothing fancy.

     Charges:

     Load *;

     SQL SELECT * FROM dbo.CHARGES;

    Store Charges into [lib://FileSystem/Charges.qvd] qvd;

    Drop table Charges;

Brandon Blaine
3 Replies
Highlighted
Partner
Partner

Weird behaviour,

 

Are you able to only load all records to memory (removing the store)?

Any error messages appear when storing to the QVD?

 

Dumb question, are there more than 3000 records on the table?

It should do quite well in loading data from SQL, i've done a few millions quite fast and storing without a problem, can you specify the exact actions are made and what goes wrong with a printscreen?

Highlighted
Contributor III
Contributor III

Hi felipedl,



Thanks for the reply. This seems to be the default regardless of if I am loading into a QVD or straight into the app from the SQL source. The table I have is ~2 million rows. It is only one table which is the result of a de-normalized query which I do because I don't believe Qlik handles complex data models very well.



I'd forgo the QVD step if I could but the data manager will not process that much data without it being in a QVD.




Brandon Blaine
Highlighted
Contributor III
Contributor III

I just had the IT team increase the number of cores on the VM from 4 to 8. I'll have to check the logs to see if this helps. The VM has 60GB of RAM.

Brandon Blaine