topic Load Data from qvd is less than expected in QlikView

Load Data from qvd is less than expected

khaled — Sat, 16 Nov 2024 03:12:59 GMT

Hello All,

I am using Qlikview R11

I have saved my data after loading into qvd file to retrieve from it later for performance purpose.

I wrote this command after the SQL command

STORE Main INTO \\BI Sales\BISales2\SalesData_Pre2019.qvd (qvd);

--538808 lines are supposed to be saved

Then I am trying to load from the qvd as follows (see attachment qlik command from sql store.sql)

Main:
LOAD
*
FROM
[\\BI Sales\BISales2\SalesData_Pre2019.qvd]
;

I am only having 1,228 lines fetched

Do you know why not all lines are retreived?

My second question how to do a load from qvd file and from SQL at the same time and combine them together

suppose that field are the same for both sources (see attachment qlikview from qvd and sql.sql)

Re: Load Data from qvd is less than expected

khaled — Sat, 29 Jun 2019 18:57:03 GMT

is there anyone can help?

Re: Load Data from qvd is less than expected

Brett_Bleess — Mon, 22 Jul 2019 19:04:16 GMT

The only thing I have is the following:

http://help.qlik.com/en-US/qlikview/April2019/Subsystems/Client/Content/QV_QlikView/Scripting/ScriptPrefixes/Buffer.htm

If that is not it, I am not sure why it is only pulling the smaller number of records, but is that number a number you recognize from a prior load at some point? If so, then the other thing to dig into here would be some sort of issue with the storage side of things I think, is the file being stored in cache and not being written to disk etc... One thing you may want to try as a test is change the path on that to local drive and see what it does that way and proceed from there... Sorry I do not have anything better for you.

Regards,
Brett

Re: Load Data from qvd is less than expected

khaled — Mon, 22 Jul 2019 19:17:44 GMT

Thanks for your help,

Do you know if I should add incremental when i save the file as a qvd?

or just add

store..... into \\.... after the SQL statement

Re: Load Data from qvd is less than expected

Brett_Bleess — Mon, 22 Jul 2019 19:33:23 GMT

I am no guru on this stuff, here is another Help link on incremental QVDs, will let you have a look to see if you may be able to use that or not...

http://help.qlik.com/en-US/qlikview/April2019/Subsystems/Client/Content/QV_QlikView/QVD_Incremental.htm

Found a Design Blog post as well that might help too with the incremental discussion:

https://community.qlik.com/t5/Qlik-Design-Blog/Overview-of-Qlik-Incremental-Loading/ba-p/1466780

Regards,
Brett

Re: Load Data from qvd is less than expected

Brett_Bleess — Mon, 22 Jul 2019 20:06:19 GMT

Something else just occurred to me, perhaps the rights/permissions on the storage location are only allowing write and not modify? That is something else to confirm, as that might explain how you got the initial load out there and the fact things are stuck on that record count. Sorry I did not think of that the first time through. Odd thing there is I would expect we should be getting an error on our side of things if that were the case though, so if that is the issue, I am a bit stumped as to how that is not causing a warning or error on things.

Regards,
Brett

Re: Load Data from qvd is less than expected

Marcos_rv — Mon, 22 Jul 2019 20:29:21 GMT

Hello, could you attach the .log of the application?
I would also like to know how you are counting the expected (538808) given that there you have repeated that in a simple table you will not see, in those cases you should use an ID.
Ahh also answering your question, you should use an incremental load, an easy way to do it would be to use a date field as a pivot.
In the first query you do a full load, the table is saved in a qvd, the next load that is made is incremental, you should read this qvd (with full load) and take the maximum date, with this you do the SQL query but Where do you use this date, which brings the data equal to or greater than that date, after this you concatenate the records you have in the qvd (full) but where you use an exists () with the ID field (which is not can repeat) so that the last load and what you had in the qvd (full) is only one table with all records, including new or modified.
You should also use a variable ... it's half complex the first time it's done but then it's all the same.

regards!!!

Re: Load Data from qvd is less than expected

khaled — Sun, 28 Jul 2019 20:47:56 GMT

hello Marcos

Can you tell me how to generate the log file ?

I count the lines from the screen while loading, I uncheck "close when finished " then I see the no of lines fetched

Re: Load Data from qvd is less than expected

Brett_Bleess — Mon, 29 Jul 2019 13:02:42 GMT

I believe the following article should help you with the logging:

How to enable Document/Script log

Regards,
Brett

Re: Load Data from qvd is less than expected

Marcos_rv — Mon, 29 Jul 2019 13:15:40 GMT

in sense:

in qmc ---> task

in QV:

settings --> document Properties