Data Load from SQL; QvOdbcConnectorPackage.exe.config Settings

[brandonblaine]

Hi,

Currently, when I load data from SQL into a Qlik Sense QVD, the Data Load Progress window shows that about 3,000 rows at a time are loaded into the QVD.

Can this be increased? The QVD is super fast obviously because it is all in memory. Loading the QVD is pretty painful though. Is there an equivalent to the SSIS fast load option?

I browsed the QvOdbcConnectorPackage.exe.config file and nothing stood out except maybe:

        <setting name="MaximumFileSize" serializeAs="String">
        <value>10MB</value> 

Here is the load script. Nothing fancy.

     Charges:

     Load *;

     SQL SELECT * FROM dbo.CHARGES;

    Store Charges into [lib://FileSystem/Charges.qvd] qvd;

    Drop table Charges;

[felipedl]

Weird behaviour,

Are you able to only load all records to memory (removing the store)?

Any error messages appear when storing to the QVD?

Dumb question, are there more than 3000 records on the table?

It should do quite well in loading data from SQL, i've done a few millions quite fast and storing without a problem, can you specify the exact actions are made and what goes wrong with a printscreen?

[brandonblaine]

Hi felipedl,



Thanks for the reply. This seems to be the default regardless of if I am loading into a QVD or straight into the app from the SQL source. The table I have is ~2 million rows. It is only one table which is the result of a de-normalized query which I do because I don't believe Qlik handles complex data models very well.



I'd forgo the QVD step if I could but the data manager will not process that much data without it being in a QVD.

[brandonblaine]

I just had the IT team increase the number of cores on the VM from 4 to 8. I'll have to check the logs to see if this helps. The VM has 60GB of RAM.